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ABSTRACT 


By using visualization best practices and embedding them in information 
technology (IT), we believe that the Department of Defense can improve its ability to 
display multi-variant information. 

The focus of this research is to design a visual information solution, based on best 
practices for displaying performance data visually, to the Electronic Military Personnel 
Record System (EMPRS). Elltimately, our goal is to improve the overall effectiveness 
and objectivity of the Navy’s selection board processes by providing a re-engineered, 
web-based, graphical solution to the visual displays currently in use by selection boards. 

The current Navy selection board voting process uses tabular forms displayed 
across five screens in a small theater-like setting. The forms are displayed very quickly 
allowing board members very little time to mentally assimilate the quantitative data 
dispersed over a wide area. In our model, we distill the data into a single graphical 
display, thus reducing the cognitive computing requirements of the board members. 

We used the Knowledge Value Added methodology to determine the proposal’s 
relative effectiveness and developed a prototype as a proof of concept. With this study 
and follow on recommendations, we foresee a considerable improvement potential in the 
Navy’s promotion board procedures and outcomes. 
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EXECUTIVE SUMMARY 


A critical issue in an officer’s eareer is the opportunity for promotion. In today’s 
technology information rich environment, it is important to re-evaluate and possibly 
change the way promotion information is presented to a seleetion board. The eurrent 
method of displaying a eandidate’s personnel reeord for promotion or selection relies 
heavily on human cognition to process tabular, quantitative information in the form of the 
Officer Summary Record (OSR) and the Performanee Summary Record (PSR) 
information. Board members use the OSR/PSR information to gain insight into the 
eandidate’s personnel data, promotion history, educational background, and job 
performance in speeific billets to evaluate each candidate for promotion potential. The 
eurrent data-to-information conversion proeess relies entirely on notes, symbols, and 
highlighted annotations on the digital summary record forms to convey the eandidate’s 
potential for promotion to the other board members. During the promotion proeess, this 
quantitative information is displayed across three to five screens in a small theater-like 
setting. The forms are displayed for two to three minutes as the individual’s record is 
reviewed by the board members. This restricted form of information presentation makes 
it diffieult to eompare candidates and may even lead to distortions in the evaluation 
proeess. 

We eonsider it essential that the board members ean quickly review and 
understand the information presented on the summary forms; information currently 
extraeted from several legaey data systems using outdated information retrieval 
processes. We see a short-lived opportunity as the Navy plans to move toward a single 
source human resource system and concurrently redesigns the Eleetronic Military 
Personnel Record System (EMPRS). Now is an appropriate time to review the adequacy 
of the tools used to present career information and if is the system developers determine 
that there is a more useful career information summary tool for selection board members, 
the tool needs to be identified early in the development of future information systems and 
integrated into the proposed designs. 
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One of the greatest benefits of data visualization is the sheer quantity of 
information that ean be rapidly interpreted if presented well (Ware, 2000). This thesis 
will offer examples of how to best transform the data into information that the board 
members ean use for optimal deeision making. Our goal is to provide a re-engineered, 
web-based, graphieal solution to the visual displays eurrently used by seleetion boards. 
The improved format will display an individual’s eareer data by graphieally displaying an 
offieer’s eareer data in a method that maximizes information flow to the briefer and 
seleetion board while minimizing seleetion board member training, reeord preparation 
time, and mental “data to information” proeessing time in the promotion evaluation 
proeess. 

This study ineludes an in depth literature review of the most eurrent visualization 
teehniques for multi-eriteria deeision-making models in eonjunetion with well established 
prineiples for the display of visual information. Mueh of this theory is based off of 
Edward Tufte’s elassieal works on the visualization of data. 

The Knowledge Value Added (KVA) methodology is used in order to measure 
the relative effeetiveness of the proposal against the eurrent system. KVA is a framework 
for measuring the value of organizations knowledge assets and provides a methodology 
for alloeating revenue and eost to an organization’s eore proeesses, based on the amount 
of ehange eaeh produees. KVA standardizes the output of all proeesses by deseribing the 
output in terms of the units of knowledge required to produee it. KVA uses a Return on 
Knowledge (ROK) ratio of the eurrent system eompared to the proposed system in order 
to eompare the measurement of the knowledge value embedded in the organization’s eore 
proeesses. 

Our main researeh question asks whether data-information visualization best 
praetiees ean be embedded in information teehnology in order to improve the overall 
effeetiveness and objeetiveness of the seleetion board proeess. We introduee a proof-of- 
eoneept (POC) prototype visualization solution through the use of Commereial Off The 
Shelf (COTS) software and teehnology in order to demonstrate the visualization eoneepts 
and eapabilities eurrently available. 


XVI 



The KVA analysis we ealeulated in this study proves that by using a web-enabled 
system and graphieally displaying the information, the proeess beeomes more efficient by 
saving time and money during each board session. For the next phase of future 
evaluation, we recommend that a side-by-side analysis using a beta system be used in 
conjunction with the old system in an actual board. By surveying the board members 
immediately after the board session, researchers could harvest immediate insights 
whether or not the new graphical displays assist in perceiving and correlating the 
quantitative information. Our study suggests that by surveying multiple boards and 
making the recommended improvements, future researchers could acquire empirical 
feedback necessary to determine whether or not the proposed system contributes or 
detracts from increasing the effectiveness and objectiveness of the overall process. This 
thesis also suggests that by reducing the preparation, briefing, and voting times for the 
board members, some of the emotional and physical strain internal to multiple decision¬ 
making within a confined time period would also be reduced. This improvement may 
well translate to more effective, and ultimately more objective decisions. 

In summary, this thesis’ primary practical focus is to make recommendations that 
will enhance the board members’ comprehension of the data which in turn, renders the 
current process more effective, efficient, and objective. The recommendations set forth 
in this study are important concepts that may contribute to immediate, short-term results 
such as shortening the selection board process while increasing its efficiency. A 
significant, long term effect includes a potential increase in retention rates resulting from 
a perceived improvement and effectiveness in the promotion or assignment systems. 


xvii 
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I. INTRODUCTION 


A. PURPOSE 

With the advancement of technology and in particular the rapid progression of the 
Internet, decision makers face an increasing problem of having too much raw information 
available. Often this problem is further exacerbated by the fact that all of the data and 
multi-variant information is not displayed in the most effective manner. As the speed and 
complexity of information greatly increases, the demand for visual tools to make sense of 
that information will only grow. By using visualization best practices and embedding 
them in information technology (IT), we believe that the Department of Defense can 
improve its ability to display multi-variant information. This thesis’s goal is to determine 
the best way to communicate more information per unit of time. In other words, what is 
the best way to transform data into information that can be quickly and accurately 
communicated effectively to others? 

The focus of this research is to design a visual information solution, based on 
best practices for displaying performance data visually, to the Electronic Military 
Personnel Record System (EMPRS) to improve the overall effectiveness and objectivity 
of the Navy’s selection board processes. The goal is to provide a re-engineered, web- 
based, graphical solution to the visual displays currently in use by selection boards, in an 
improved manner and format that will display an individual’s career data. By maximizing 
information flow to the briefer and selection board and minimizing selection board 
member training, record preparation time, and labor-intensive “data to information” 
processing time in the promotion evaluation process, the ultimate re-design should be 
clear, concise, comprehensive, and entail a scalable methodology allowing for changing 
performance dimensions. An additional objective is to propose the Knowledge Value 
Added (KVA) methodology as a metric by which to gauge or measure this solution 
against the current process in order to determine its relative future effectiveness. 

1. Scope 

This thesis is primarily concerned with the graphical representation of information 
to maximize information flow about an individual’s promotion/selection potential while 
minimizing the amount of effort the user expends extracting this information. Emphasis 
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is placed on the evaluation of the human faetors involved in the effeetive display of 
visual information, to include size of display, colors used in the display, and effeetive 
ways to present information. The eurrent form uses only text and grid lines. Included in 
this study is an evaluation of other ways to present information, such as with charts or 
graphs, to more effectively eonvey the information reviewed on the summary form. 

The vision of this project as articulated by the Navy Personnel Command (NPC) 
is that of a two-phased implementation with the first phase being the incremental step of 
re-designing the visual displays to look at today’s “Report Card” (OSR/PSR data). This is 
the phase that our thesis addresses. The seeond phase is a radieal re-design of the displays 
to eneompass “Tomorrow’s Five Veetor Model” in the advaneement and future 
promotion of Naval personnel. The seope of the two phased implementation is as follows: 

a. Naval Postgraduate School (NPS) Project Scope 

The current displays are used by seleetion boards, detailers, and other 
eareer planners to determine the future promotion, advaneement, and assignment of Navy 
personnel. In an attempt to limit the seope of this project, the authors have deeided only 
to use the officer statutory selection board process example in whieh to base the study 
and proof of eoneept model. Our intent is that this system will evolve into a true web- 
based interaetive model, however, at the present time this web-based format has yet to be 
identified, so we have ehosen to develop a proof of eoneept model using Maeromedia 
Dreamweaver MX for our web applieation, utilizing a three tier arehiteeture 
eneompassing a notional Mierosoft Aeeess relational database. Due to the limited time 
and resources available for this research, we have deeided to expend minimal effort on 
explaining the eomplexities of the current network and data base arehitectural 
teehnologies that exist and that are eurrently being re-engineered through the Navy’s 
personnel and manpower modernization effort. 

b. Navy Personnel Command (NPC) Project Scope 

The Navy Personnel Command’s foeus and scope states that the re¬ 
engineering effort must design a visual display with the following oharaeteristies: 
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• Compatible with the Eleetronie Military Personnel Records Management 
System (EMPRS) or the technical refreshed version of EMPRS currently under 
development 

• Compliant within the NMCI network standards and operate successfully within 
enclave 

• Should have compatibility with PeopleSoft HRMS applications 

• Operational within Wood Hall Bldg. 769, Navy Personnel Command 
(NAVPERSCOM) 

• Will provide executable design for selection boards 

• Will not alter current processes or require changes to U.S. Code title 10 
promotion law, or career milestones 

• Will not change manpower requirements for operations and support 

• Increases the relative effectiveness of visual displays 

• Provides for a graphical representation of OSR/ESR (Officer/Enlisted 
Summary Records) data reflecting performance history, awards, and promotion 
progression set against a career timeline or history of assignments. 

• Integral into the project portfolio of Sea Warrior 

• Successfully extracts data for the Eive Vector Model (5VM) from the Navy 
Personnel Database (NPDB) and the Navy Standard Integrated Personnel 
System (NSIPS). 

2. End State 

The primary practical focus of this research is to make recommendations in order 
to enhance the promotion board members’ comprehension of the data in a way that makes 
the current process more effective, efficient, and objective. This is an important factor 
that may contribute to immediate short-term results, such as shortening the selection 
board process while increasing its efficiency. Eong term effects could include an increase 
in retention rates due to the enhancement of personnel satisfaction attributed to a more 
effective determination of the future promotion or assignment of Navy personnel. The 
overall vision is for a two-stage implementation with the final stage using the enhanced 
displays for the projection of the Eive Vector Model for promotion purposes in the future. 

The concept of the Eive Vector Model is to show Naval personnel their career 
path, identify the requirements they need to complete, and to qualify them for future 
assignments. The model aligns professional development requirements with certifications 
and qualifications (military and civilian) that are recognized as the industry standard. 
Once the Elect has identified the requirements, subject matter experts (SMEs) in each 
field plot them on the vector model where an individual should ideally meet those 
milestones in their career. The model also considers personal development, leadership. 
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management, and individual performance. (Perron). The ultimate end state and radical re¬ 
design would then be to use the first phase re-design of the visual displays to show the 
Five Vector Model as a way of displaying the qualifications and individual performance 
with regard to promotion. 

The immediate benefit of this study, however, will be an improved process for 
effectively and efficiently selecting and promoting Naval officers by improving the visual 
representation of performance information for the selection boards while minimizing 
selection board member training, reducing record preparation time, and reducing the 
mental “data to information” processing time. 

B, BACKGROUND 

The Performance Summary Record (PSR), Officer Summary Record (OSR) and 
Enlisted Summary Record (ESR) are used by selection boards, detailers, and other career 
planners to determine the future promotion, advancement and assignment of Navy 
personnel. During the selection board process, two or three different forms are displayed 
across three to five screens in a small theater-like setting referred to as the “tank.” These 
forms are displayed for two to three minutes as the individual’s record is reviewed by the 
board membership. It is essential that the board members quickly review and understand 
the information presented on the summary form. The summary forms are currently 
designed to capture a combination of basic personnel information including designator or 
rating, year group, education history, and special qualifications. The form also includes a 
listing of duty assignments and a numeric summary of performance marks that the 
individual under consideration has received throughout his or her career. These forms are 
annotated by selection board members and used to brief other selection board members 
on the individual’s career. The personnel data displayed on the forms are predetermined 
and there is no ability to tailor the form or the information on the form for specific types 
of boards. When special career information is required for a particular type of board, that 
information must be manually entered on the form by selection board administrative 
staff The information on the forms is extracted from several legacy data systems using 
outdated information retrieval processes. 
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1, Current Process 

The current method of displaying a candidate’s personnel record for promotion or 
selection decisions relies primarily on human cognition to process tabular 0/ESR and 
PSR data into the required information used for comparison and decision-making. Board 
members use Officer/Enlisted (0/E) Summary Record Eorms and Performance Summary 
Records to gain insight into the candidate’s personnel data, promotion/advancement 
history, educational background, and job performance in specific billets in order to 
evaluate promotion or selection potential. Applicable portions of the candidate’s record 
on microfiche are used to add more detail to the performance, award, education, and 
special qualifications entries made on the summary records. A system to provide the 
candidate’s data, called the Electronic Military Personnel Records System, has been 
implemented and is explained in further detail in chapter II. 

2. Need for Improvement 

This current data-to-information conversion process relies entirely on notes, 
symbols, and highlighted annotations on the digital summary record forms to convey the 
candidate’s potential to other board members for promotion or selection decisions in a 
short three minute or less presentation. The format in which to view this information 
currently is limited and burdensome because it is made up of primarily quantitative data 
dispersed widely across five different display screens, thus forcing board members to 
search for and extract relative pieces of data and then mentally process and integrate the 
data into useful information. This restricted form of information presentation makes it 
difficult to compare candidates at a minimum, and may conceivably even lead to 
distortions in the evaluation process. 

C. RESEARCH QUESTIONS 

This thesis will attempt to answer the principle research question of whether data- 
information visualization best practices can be embedded in information technology in 
order to improve the overall effectiveness and objectiveness of the selection board 
process. Additionally, will integrating web-based technology with the visualization 
techniques reduce the time required to prepare, brief, and make decisions in the 
promotion process? Can web-based data visualization technologies serve as an enabler to 
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this problem, and if so, what are the relevant eriteria for deeiding whieh teehnologies are 
the most appropriate in this regard? What is an appropriate user interfaee design for this 
deeision problem? 

In order to answer these questions, our researeh will explore subordinate and 
more speeifie questions sueh as whieh prineiples of graphieal praetiee apply to this type 
of data and which will best convert the data into meaningful visual information in this 
decision context? Are there colors, layering/separation, or multidimensional schemes 
that would facilitate information flow without inserting bias in envisioning a candidate’s 
personal and performance information? We will also introduce the Knowledge Value 
Added methodology which will provide us with a performance ratio for all the core 
processes involved in order to determine if there is any increase in effectiveness between 
the “as-is” and “to-be” models. 

D. METHODOLOGY 

1. Proof of Concept Context 

The proof-of-concept context for this research is the current Naval officer 
promotion/selection process. In order to answer the principle research question, the 
authors have developed a proof-of-concept prototype visualization solution through the 
use of Commercial off the Shelf (COTS) software and technology in order to demonstrate 
the visualization concepts and capabilities currently available. This prototype uses an 
artificial database and simulates the current system’s input and outputs. Using three-tier 
architecture, the prototype extracts the data from the database and presents an improved 
visual display for the board to interpret and compare. In order to achieve the research 
goals, we have completed the following analysis and prototype development: 

• An “as-is” analysis with a Return on Knowledge (ROK) spreadsheet and 
graphical representation. 

• A “to-be” incremental model spreadsheet. 

• A fully functional web site that utilizes a notional relational database. 

• Prototype web-based charts and graphs using different colors, along with 
layering and separation multidimensional schemes that would facilitate 
information flow. 
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2, Literature Research 

In order to gain extensive insight into the most effeetive and objeetive methods of 
displaying quantitative information, the authors have eondueted an in depth literature 
review of the most eurrent visualization techniques for multi-criteria decision-making 
models in conjunction with well established principles for the display of visual 
information. Much of this theory is based off of Edward Tufte’s classical works on the 
visualization of data. 

Additionally we have conducted a detailed study of the Knowledge Value Added 
methodology and have primarily focused our area of research on works of KVA’s 
foremost authority and co-originator, Dr. Thomas Housel. 

E, ORGANIZATION OF THESIS 

This thesis is organized into five major sections or chapters. Chapter I is the 
overview of our research. Chapter II is primarily devoted to explaining the promotion 
board process in detail. This chapter is mainly for the reader with little or no background 
in the Naval Promotion Board process. Chapter III reviews the theory and literature used 
in the research. Chapter IV clarifies the details of the Knowledge Value Added 
methodology used to compare the current and future system. Chapter V will introduce 
our working prototype. Finally, Chapter VI will follow up with a conclusion, some 
recommendations, and possible areas for further research. 
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II. PROMOTION BOARD PROCESS 


A. OVERVIEW 

This chapter will help the reader to understand the promotion proeesses and 
procedures that are detailed in the remainder of this doeument, partieularly the literature 
review in ehapter III. It is particularly important for the reader to have a solid 
understanding of the proeess before we explain our proof of concept methodology in the 
later chapters. We briefly outline the promotion board proeess as it exists today and eover 
the basics of the proeess. With this chapter, we intend to inform the reader who has not 
had any prior experience with, or knowledge of, the Navy’s promotion board proeess. 

There are two types of boards: Statutory and Administrative. Statutory boards 
include promotion (including special and spot promotion boards). Selective Early 
Retirement Boards (SERB), and various continuation boards. 

All other boards are administrative. Statutory boards are governed by law, 
primarily Title 10 of the U.S. Code. Administrative boards are governed by instruction or 
policy. Statutory boards are eonvened by the Seeretary of the Navy (SECNAV) and the 
results are approved by the President, Seeretary of Defense (SECDEE), and SECNAV. 
Administrative boards are convened by the Chief of Naval Personnel (CNP) or the 
Commander, Navy Personnel Command (CNPC) and the results are approved by 
CNP/CNPC. 

The preeept is a doeument that is signed by the convening authority and direeted 
to the president of the board, giving general and specific guidance to the board regarding 
the criteria upon which their selections should be based. The precept provides several 
important faetors for the board to eonsider, such as board membership, promotion 
opportunity-stated pereentage (i.e. 95 percent, 80 percent, etc), and any specific guidance 
sueh as the special needs of the Navy at that time. The preeept is the only guidanee for 
seleetion provided to a board. Using the preeept as guidance, the only other sources of 
information about an officer allowed for consideration by a statutory board are the 
following: 
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• Personnel Record Microfiche - contains an officer’s fitness reports 
(FITREPS), photograph, personal awards and other matters of official record. 

• Officer Summary Record (OSR)/Performance Summary Record (PSR) - an 
officer’s career resume containing a summary of his or her microfiche. 

• Any correspondence the officer submits to the board about their record. 

No information other than what is listed above is allowed to be discussed or presented 
before a board. 

In an effort to narrow the scope of this project, we decided to focus our effort and 
base our proof of concept solely upon the officer statutory board process. In particular, 
we focused on the promotion boards because they are the most common by far and carry 
the largest impact. 

The mission of any board is to select those officers that are “best qualified” based 
on performance. Additional guidance in the precept addresses specific guidance for the 
board to consider in determining the best-qualified officers. The rest of the chapter will 
illustrate the typical promotion board process and will be broken down into three 
sections: 

(1) Pre-board 

(2) Record Review 

(3) Selection. 

1, Pre-Board Phase 

All boards take place at the Navy Personnel Command (NPC) in Millington, 
Tennessee. The initial groundwork for a promotion board begins about four months 
before its convening date at NPC when the initial list of eligible officers is compiled and 
modified as required. The eligible list is continually synchronized with an official 
automated database to ensure consideration of all candidates and the master file is 
queried six weeks prior to the board convening for FITREP continuity. Messages are then 
sent out for any missing FITREPs. The images that are reviewed by selection boards, and 
the data that is presented on the OSR/PSR, come from two different sources and there are 
often errors and inconsistencies between the two. Researching and correcting those errors 
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and inconsistencies is performed by the assistant recorders who arrive and begin their 
work the week before the board convenes. This process is called “prep week”. 

Board members arrive at NPC on the date authorized in the precept and convene 
the board. Board membership is secret until the board convenes and members are 
specifically directed not to visit detailers prior to or during board deliberations. The 
process is designed to ensure a level playing field for all involved. Above all, board 
membership is carefully balanced to represent the demographics of all the candidates in 
the selection zones. The precept is discussed and the board begins work on the individual 
record review. 

2. Records Review Phase 

All of the eligible officers’ data or records are divided up and impartially 
dispersed among the voting members of the board for review. Promotion board members 
review a digitized form of the official record for each eligible officer. Within each 
eligible officer’s digitized record, the board members review the officer’s photograph, 
FITREPs, OSR/PSR, and personal awards starting from the Navy Achievement Medal 
and higher. Education, promotion history, and board certification are some of the other 
areas of interest being carefully looked at during this phase. Additionally, the board 
members are provided and privileged information (formerly Fiche 5) on all eligible 
officers, if applicable. This information may be medical or punitive, such as medical 
board information or non-judicial punishment awarded at Article 15 proceedings and 
courts martial results. Most officers, however, do not have any privileged information in 
their record. 

The board members are provided the OSR and PSR which are administrative tools 
used to track individual performance. Since 1995, the PSR has replaced the OSR as the 
sole performance-tracking tool. The PSR provides a quick snapshot of the eligible 
officer’s education, duty assignments, awards, and FITREP marks, while excluding the 
narrative section of the FITREP. An additional source of information reviewed by the 
board is correspondence sent by the eligible officer in the form of a letter directly to the 
president of the board. 
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The Electronic Military Personnel Records System (EMPRS) is the principal 
information tool used by the board members to review a candidate’s record. EMPRS is an 
image-based information management tool that is used for processing, storing, and 
distributing both digital images and ASCII data. The EMPRS encompasses three 
distinctive modules which are the Primary Input/Output system (PRIO), the Eitness 
Report/Evaluation system (EITREV) and Selection Board Module (SEEBD). These 
modules work together to create a system that allows operators to manage member 
records from the desktop and for selection board members to electronically evaluate 
personnel for promotion. 

Specifically, the board members will sit at an EMPRS terminal and electronically 
review all of the pertinent data previously compiled and downloaded on his or her 
assigned candidates. The board members have the ability to not only review but also 
highlight, mark up, and annotate the PSR from the EMPRS terminal by means of a 
handwriting tool to draw lines, circles, and various other pre-configured auto shape 
functions that allow the reviewer/briefer the ability to highlight and specifically point out 
to other board members pertinent pieces of information from that record (Eigure 1). 



Eigure 1. OSR/PSR Cover Page displayed on EMPRS. (Prom BUPERS Presentation) 
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The board carefully considers, without prejudice or partiality, the record of every 
eligible officer. The officers selected are those whom a majority of the members of the 
board consider best qualified for promotion, giving due consideration to the needs of the 
Navy for officers with particular skills. In addition to the standard of best qualified, all 
officers recommended for promotion must be fully qualified; that is, each officer must be 
capable of performing the duties of the next higher grade. 

3. Selection Phase 

After the records review phase, the board moves on to the next step - the selection 
phase. In this phase, the board members move into a room called “The Tank” which is a 
private, theatre-like room where all the members discuss and vote on candidates. The 
annotated OSR/PSRs are projected onto five large screens in the tank, (Figure 2) and 
each record is briefed by the board member who reviewed the candidate’s record. 

After the briefing officer has discussed the candidate and all questions have been 
asked and answered, each member uses a “secret ballot” computer keypad located on the 
arm of their seats to cast a confidence level vote for the selection of the candidate. Each 
member can vote either 100 percent (the member is 100 percent sure the candidate should 
be selected), 75 percent, 50 percent, 25 percent or 0 percent (the candidate should not be 
selected ). After all of the votes are cast, the computer in the tank combines them into an 
overall confidence rating which is then displayed as an average percentage on a monitor 
for all the board members to see. 

The confidence rating of each candidate is recorded and then ranked after all the 
records have been reviewed. A board member (usually the senior member or president of 
the board) will propose an option for selection to the entire board. The board selects a 
number of the records from the top scorers to be “tentatively selected” or what we’ll call 
the TS category. The board will either vote on the motion or members will offer counter¬ 
proposals. Whichever proposal is accepted, it is accepted by a majority vote of all the 
members. This same scenario is repeated when the board attempts to determine which 
number of the bottom scoring candidates should be “dropped from further consideration” 
and those candidates will be placed in the DEC category. All of the candidates between 
the TS and the DEC categories fall into what is called the “crunch” category or zone. 
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Each of the candidates that fall into the “eruneh” eategory is reviewed by a 
different board member than the one who originally briefed their record. Each candidate 
receives another review and brief in the tank and the process starts over again and is 
repeated until eventually the entire eruneh list is diminished and all eandidates have 
either been seleeted or dropped from further consideration. Several “tank” sessions are 
usually required before the board selects the final number of eandidates they believe are 
best qualified for promotion. Einally, the board then completes its deliberations and votes 
to confirm the tentative seleetions. 

The following sections will detail exactly what the board is looking at when 
voting in the “tank.” This will give the reader a better understanding of how and why we 
designed our proof of eoneept model. 



Eigure 2. OSR/PSR info displayed in the “tank”. (Erom BEIPERS Presentation) 

B. PERFORMANCE SUMMARY RECORD (PSR) 

The following seetion will discuss how the boards use the Offieer Summary 
Record (OSR)/Performance Summary Record (PSR) to make their seleetions. The 
OSR/PSR is an administrative tool that summarizes an offieer’s professional and 
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performance history and serves as the boards “snap shot” of the candidate’s career. The 
board member who reviewed and annotated the candidates data on the EMPRS will brief 
that record in the “tank” and is the annotated OSR/PSRs that he or she is summarizing 
to the other board members. The OSR consists of the cover page (Figure 3) and FITREP 
detail pages that document the old FITREP system reports. The PSR consists of the new 
system reports. Fet’s take a look at a sample OSR/PSR and look at this notional record as 
a selection board member would do during a board. In an effort to keep this concise and 
to limit the scope of our proof of concept model, we will be reviewing only the new 
FITREP evaluation page (PSR) and not the old FITREP evaluation page (page 2 of the 
OSR proper). 

1. OSR/PSR Cover Page 

When a board reviews the OSR/PSR data they start with the cover page. The 
OSR cover page consists mainly of administrative data such as the candidates name, 
social security number, and career history. The comments in our example (Figure 3) are 
typed on the OSR/PSR image shown on the briefer’s EMPRS screen and the circles and 
lines are drawn using computer graphics. The briefer (the board member who reviewed 
the candidate’s record) made these remarks to highlight important points in the 
candidate’s career to the other board members when they are voting in the tank. 



Figure 3. OSR/PSR Cover Page EMPRS view. (From SUPERS Presentation) 
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Apart from the standard information (name, SSN, date of birth, ete.) at the top, 
notiee that the briefer has annotated several items that he deemed important to point out 
to other board members. For example, we ean see that the briefer points out this 
candidate as an E-2 C.O. and was “ALWAYS ABOVE THE PACK, COMMUNITY 
LEADER, RECOMMENDED EOR ERS C.O. BY BERNIE SMITH.” We can also see 
that another briefer (different color marks) noted and circled comments such as CMD, 
E2C EXPERT, POST GRAD, AND 1%, all of which could help the other board 
members with their decisions in the tank. 

2. PSR 

The following is a brief description of how the “new system” EITREPs are 
displayed in the PSR. The new EITREP system took affect in August 1995 and 
completely changed the EITREP and the look of the PSR. Prom Pigure 4 we can see from 
looking at the top line and reading across from left to right, that this individual’s present 
grade (PG) is 05. We can also see that he was the Commanding Officer of VAW-126 and 
that this particular reporting period (the top line) was from 090195 to 041296. The report 
covers a 7 month period and his reporting senior who wrote this report is T.E Zelibor 
whose present grade is 06 and his title at the time of the report was Commander (COM). 

The remainder of the columns across the page contain the individual numbers and 
grade averages that make up the meat of the report. Por trait grades, this candidate 
received two “4.0” grades and five “5.0” grades. His individual average was 4.71 and the 
reporting senior’s (RS) trait grade average for this summary group was 4.60. This 
candidate was in a summary group (competitive category) consisting of 6 individuals 
(add 3, 1, 2 reflected in the summary line of the promotion recommendation column). 
The total number of Commanders (05 rank/grade) for which Capt Zelibor was the 
reporting senior, regardless of designator (Active, and Reserve, Line and Staff), was 7 
(Top number in CUM column). Capt Zelibor’s cumulative average for the 7 Commanders 
he had evaluated up to the end date of this report, (including the 6 in this summary group) 
was 4.61 (Bottom number in the CUM column). This individual was given a promotion 
recommendation of “Must Promote” (X under “MP”) along with one other in the 
summary group. Three individuals in the summary group were given “Promotable” 
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recommendations (3 under the “PR”) and two were given “Early Promote” (2 under 
“EP”) promotion recommendations. The report was a regular (“RG”) report, which 
maintained his record continuity (concurrent and Operational Commander Reports do not 
count for continuity). 

Prom Pigure 4 we can see that the two briefers who reviewed this record on the 
EMPRS noted that this individual was “AEWAYS ABOVE THE PACK AS C.O. AND 
AEWAYS RANKED.” Also annotated was the fact that he was a “CLP SILVER 
ANCHOR WINNER” during the top reporting period. The heavy line drawn across the 
middle of the report is just separating two different Commands. 
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Pigure 4. PSR with briefers’ annotated marks. (Prom BUPERS Presentation) 


3, Summary 

In this chapter we have reviewed the Promotion Board process with the intent to 

explain the basics of the procedures to the reader who has had little or no prior 

knowledge of, or experience with, the Navy’s promotion process. We believe this will 

provide a solid foundation from which the reader may better derive a thorough 

understanding of our methodology and proof of concept model. From the last section it 

should be fairly clear to the reader why we believe there could be a more effective way in 
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which to display the OSR/PSR information to the board members in the tank. Mueh of 
the pertinent information is quantitative, eonvoluted, and widely dispersed aeross the five 
sereens in the tank. The following ehapters will explain the theory behind our proposal, 
an eeonomie eomparison of the eurrent system to what we are proposing, and a look at 
the proof of eoneept prototype followed by the eonelusion and reeommendations. 
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III. LITERATURE REVIEW 


A. INTRODUCTION 

The current method for displaying a candidate’s personnel record for promotion 
or selection decisions relies heavily on human cognition to process tabular OSR/PSR data 
into the required information used for comparison and decision making. The majority of 
the information is numerical and is often spread out across different places among the 
five large screens within the tank. Our challenge is how can we arrange the data 
graphically or visually so that it is aesthetic to the viewing board members in order to 
allow them to more easily correlate all of the data quickly and objectively? Can we make 
it web-based so that some of the extremely time consuming manual processes such as 
scanning documents onto the system or having to separately download data into the tank, 
be eliminated completely? How do we effectively measure this to see if it’s worth the 
effort to redesign? In other words how do we go about measuring the change in 
performance from the “as is” baseline process in use today against our proposed “to be” 
model of tomorrow? 

This chapter will review the literature we used in order to answer the previous 
questions. Section B discusses the Knowledge Value Added (KVA) methodology that 
was developed by Dr. Thomas J. Housel and Dr. Valery Kanevsky. KVA is essentially a 
performance metric that provides a means for objective measurement of the relationship 
between knowledge and value in organizational processes. Section C will detail the best 
practices for effectively displaying quantitative information graphically as seen by one of 
the renowned authorities in that field, Edward R. Tufte. 

B. KNOWLEDGE VALUE ADDED (KVA) 

Omar A. El Sawy, in his book Redesigning Enterprise Processes for e-Business, 
explains that one of the central tenets of the Knowledge Value-Added (KVA) 
methodology is that: 


19 



When a process is redesigned to be more knowledge-creating, it is often 
hard to measure the added value to the customer of the process. One of the 
characteristics of such a redesign principle is that it causes the business 
process to learn more from the participants it interacts with each time there 
is an interaction. Thus the value of the process increases over time as 
intellectual capital through repeated execution, even though initially there 
is no immediate added value for the customer. It is difficult at the design 
stage to measure what the return on redesigning the process might be. The 
quicker that the new knowledge in the process can be translated into value 
that the customer is willing to pay for (whether through this business 
process or a related one), the more effective the business process design is 
(El Sawy, 2001). 

How can we adequately measure the effectiveness of what we’re proposing? It 
would be very difficult, to measure whether or not the board members were making 
better decisions with the redesigned process. Although the redesigned process would 
result in shorter board times and thus resulting in less costs, we would be remiss if we 
were to base the effectiveness solely on time savings. Much of the change that we need to 
measure is intrinsic and may not be immediately apparent. For example, better decisions 
made in the promotion and assignments processes of today may lead to an improved 
quality of life and higher retention rates for tomorrow. 

1, Knowledge Value Added Theory 

The method we chose to measure the benefits of greater efficiency in the decision 
process is the Knowledge Value Added (KVA) methodology. This methodology is 
beneficial to non-profit organizations because it presupposes knowledge as a surrogate 
for value and therefore can be used independent of profit generation. Because of this and 
the fact that cutting costs and reducing head count were not viable options for our proof 
of concept, KVA was chosen because it enables managers to measure the performance of 
corporate or community knowledge assets whether the knowledge is deployed in 
information technology (IT) or resident within employees’ heads. 

KVA analysis produces a Return on Knowledge (ROK) ratio to estimate the value 
added by given knowledge assets regardless of where they are located within the 
organization. The essence of KVA is that knowledge utilized in organizations’ core 
processes is translated into numerical form. This translation allows allocation of revenue 
in proportion to the value added by the knowledge as well as the cost to use that 
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knowledge. Traeking the eonversion of knowledge into value while measuring its bottom 
line impacts enables managers to increase the productivity of these critical assets (Housel 
and Bell, 2001). 

KVA provides a methodology for allocating revenue and cost to an organization’s 
core processes based on the amount of change each produces. KVA standardizes the 
output of all processes by describing the output in terms of the units of knowledge 
required to produce it. Notably, the knowledge required to make these changes is a 
convenient way to describe the conversion process. Knowledge is the know-how required 
to produce process outputs, process instructions, or in our case, process decisions. This 
kind of knowledge is proportionate to the time it takes to learn it. 

2. Knowledge Value Added Method 

The KVA method is extremely beneficial to organizations because it is simple 
enough to be applied in seven steps yet it is robust enough to produce a desired level of 
granularity should the organization desire a more comprehensive view of their processes. 
Housel and Bell offer three different ways to establish the value of knowledge embedded 
in the organizations core processes (Table 1). 


Steps 

Learning time 

Process description 

Binary query method 

1. 

Identify core process and its subprocesses. 

2. 

Estabiish common units to 
measure learning time. 

Describe the products in terms 
of the instructions required to 
reproduce them and select unit 

Of process description. 

Create a set of binary yes/no 
questions such that all possible 
outputs are represented as a 
sequence of yes/no answers. 

3. 

Calculate learning time to 
execute each subprocess. 

Calculate number of process 
instructions pertaining to 
each subprocess. 

Calculate length of sequence of 
yes/no answers for each 
subprocess. 

4. 

Designate sampling time period long enough to capture a representative sample of the core 
process's final produtS/service output. 

5. 

Multiply the learning time for 
each subprocess by the num¬ 
ber of times the subprocess 
executes during sample period. 

Multiply the number of process 
instructions used to describe 
each subprocess by the number 
of times the subprocess 
executes during sample period 

Multiply the length of the yes/no 
string for each subprocess by the 
number of times this subprocess 
executes during sample period. 

6. 

Allocate revenue to subprocesses in proportion to the quantities generated by step 5 and calculate 
costs for each subprocess. 


7. Calculate ROK, and interpret the results. 


Table 1 Three Approaches to KVA (From Housel and Bell) 
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Although the binary query method is usually the most preeise method, it is also 
the most time intensive and is usually only used for situations requiring a high degree of 
aeeuraey and granularity. The method we ehose for our Proof of Coneept is the Learning 
Time method. This method allows those who use it to establish rough-eut estimates of the 
value of knowledge within proeesses. It ean be aecomplished more quiekly than the 
binary query method and is targeted at the aggregate level of analysis. An example of a 
high-level aggregate KVA analysis is shown in Table 2. 
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Total 


100 

1.7100 


37,020 

90,045 

100% 

$242.0 




Table 2 High-level Aggregate KVA Analysis (From Housel and Bell) 


Table 2 shows an example of a eompany’s results of a seven step high-level KVA 
analysis. Right away managers are able to determine the relative performance of the core 
functional areas in terms of ROK. The results would serve as a starting point from which 
a more detailed KVA analysis could be done to identify knowledge embedded in sub¬ 
processes and to help managers make improved decisions on how to increase the 
company’s profits. 

Learning Time (LT) has been established as a fast and appropriate way to 
measure the amount of knowledge contained in any given process. The amount of 
knowledge embedded in a process corresponds to the amount of time necessary for an 
average person to learn how to complete the process properly. The knowledge can also be 
represented by the process instructions required to produce the process output effectively. 
Learning time is in proportion to the amount of knowledge learned. In this manner, 
learning time can be used as a common-sense indicator of the amount of knowledge 
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embedded within a given proeess. Usually the Subjeet Matter Experts (SME) for a 
particular process can provide actual realistic estimates of the learning time required for a 
given process based on formal and informal training times, experience on the job, 
employee interviews, training manuals, and programs. 

3. KVA Example 

The following is a short and extremely simple example of KVA and how it works, 
taken from Housel and Bell’s Measuring and Managing Knowledge. In the Widget 
Company, there is one person, the owner, who makes and sells widgets. This person 
knows all there is to know in order to make and sell widgets for $1. The owner’s sales- 
production knowledge can be used as a surrogate for the dollar of revenue generated by 
his application of the core process knowledge. We can determine how long it takes the 
widget company owner to transfer all the necessary sales and production knowledge to a 
new owner. Eurther, we can use these leaning times to allocate the dollar of revenue 
between the sales and production processes. Eor simplicity sake lets say that it takes 100 
hours for someone to learn a new process with 70 hours spent learning how to make the 
widget and 30 hours learning how to sell it. This would indicate that 70 percent of the 
knowledge and value added was contained on the production process and 30 percent in 
the sales process. It would follow that $.70 of the revenue would be allocated to 
production knowledge and $.30 to sales knowledge. 

All that would be left to do in this example would be to determine how much it 
costs to use the sales and production knowledge and then we would have a ratio of 
knowledge value added to knowledge utilization cost. In other words, we can measure 
return on knowledge (ROK). Eor the sake of argument, we will assume that the total cost 
to sell and produce a widget was $.50; with $.25 for sales and $.25 for production. The 
basic approach here is to find out how much it costs to use the sales and production 
knowledge. In this case, the cost is directly tied to how long the new owner spends 
performing each process. As it turns out, in this case, the new owner spends the same 
amount of time to do both and, therefore, the cost to use the knowledge of each process is 
the same. 

Based on the estimates for the distribution of revenue and cost, we would generate 

an estimate of the ROK. We would conclude that the production process is a more 
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productive use of the knowledge asset (ROK = .7/.25=280%) that the sales process 
(ROK=.30/.25 = 120 percent). 

4, KVA Summary 

KVA provides the organization with a way of making decisions on how best to 
shift or eliminate knowledge as in the case of whether or not to automate a core process. 
The key issue, and one that applies to our case, is how to redeploy knowledge from 
people and procedures or work rules in to IT so that it can be executed more rapidly, 
therefore more often, and at a lower cost. If the ROK of a given process is not improved 
with the implementation of the automation, then steps must be taken to improve its 
functionality and performance. 

The same view may be extended to estimate the amount of knowledge contained 
in IT. To do this, the organization must choose IT outputs within the core process and 
then ask the resident SME to estimate the time necessary for learning how to generate the 
same outputs. In other words, he or she should estimate how long it would take to do the 
same process without the IT. How long would it take to teach the average person to 
produee the same output manually without the IT? Most likely, it would take much 
longer for the average person to produce the same outputs but what really needs to be 
determined is how long it would take that person to learn to produce the outputs (i.e. the 
learning time estimate). 

In this way, IT is just another layer of proeess knowledge. KVA also allows the 
organization to determine that important question of what is its return on IT? KVA 
allows the organization to obtain a true snapshot of the return it’s getting from IT. 
Because the numerators and denominators come from different data sources (real 
revenues and costs) managers won’t be able to manipulate cost (i.e. the denominator) to 
obtain the numerator (e.g. cost savings or cost avoidance) so that a false return would be 
generated. 

C. BEST PRACTICES FOR VISUALIZATION TECHNIQUES 

What we see is not always what we think we see. We must ensure that 

what we portray is not misleading or misunderstood, and that we are 

conveying the intended message (Brown, 1995). 
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This section will explore the current best practices for displaying quantitative 
information and make recommendations on how to most effectively and objectively 
display the OSR/PSR information. One of the greatest benefits of data visualization is the 
sheer quantity of information the audience can rapidly interpret if the visual data is 
presented well. We have chosen to follow the theoretical research of Edward R. Tufte 
because of his renowned expertise in this field. 

This chapter outlines Edward Tufte’s pioneering work on the use of visual 
graphics to display quantitative information and it mainly consists of text and ideas taken 
from three of his books on this subject matter. We have included a few diagrams as 
examples but the reader should examine the books in order to get a full understanding 
and appreciation of the graphics and concepts he introduces. 

Modern data graphics can do much more than simply substitute for small 
statistical tables. At their best, graphics are instruments for reasoning 
about quantitative information. Often the most effective way to describe, 
explore, and summarize a set of numbers - even a very large set - is to 
look at pictures of those numbers. Eurthermore, of all methods for 
analyzing and communicating statistical information, well-designed data 
graphics are usually the simplest and at the same time the most powerful 
(Tufte, 2001). 

1. Graphical Principles 

In his book The Visual Display of Quantitative Information, Edward R. Tufte 
explains that excellence in statistical graphics consists of complex ideas that are 
communicated with clarity, precision, and efficiency. He further notes that graphical 
displays should: 

• Show the data. 

• Induce the viewer to think about the substance rather than about methodology, 
graphic design, the technology of the graphic production or something else. 

• Avoid distorting what the data have to say. 

• Present many numbers in a small space. 

• Make large data sets coherent. 

• Encourage the eye to compare different pieces of data. 

• Reveal the data at several levels of detail, from a broad overview to the fine 
structure. 

• Serve a reasonably clear purpose: description, exploration, tabulation, or 
decoration. 

• Be closely integrated with the statistical and verbal descriptions of a data set. 
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In order to gain the most benefit from redesigning the promotion board displays 
so that the data is elear, easily understood, and not biased in any manner, we adhered to 
Tufte’s principles of graphical excellence. These principles are as follows: 

• Graphical excellence is the well-designed presentation of interesting data - a 
matter of substance, of statistics, and of design. 

• Graphical excellence consists of complex ideas communicated with clarity, 
precision, and efficiency. 

• Graphical excellence is that which gives to the viewer the greatest number of 
ideas in the shortest time with the least ink in the smallest space. 

• Graphical excellence is nearly always multivariate. 

• Graphical excellence requires telling the truth about the data. 

a. Graphical Integrity 

Tufte’s works are appropriate for our study because he has developed a 
consistent approach to the display of graphics which enhances its dissemination, 
accuracy, and ease of comprehension. Most importantly, he stresses that you cannot gain 
graphical excellence without telling the truth about the data which is paramount in our 
proposal for the redesigned promotion displays. 

He explains that the problem lies in presenting large amounts of 
information in a way that is compact, accurate, adequate of the purpose, and easy to 
understand. Specifically, a designer must show cause and effect, insure that the proper 
comparisons are made, and achieve the valid goals that are desired. A major problem 
with presenting the visual display of quantitative information is the misperception of that 
data by the viewer. Different people see the same objects somewhat differently and 
context-dependent perceptions change with experience . Another factor that could distort 
the data involves miscommunication. A person’s perception of an object can change 
depending on what others have already said. 

In order to reduce these effects, the designer needs to strive for uniformity 
in graphics and assurance that the representation accurately depicts the target numbers. 
Tufte mentions two goals that if followed, may alleviate these problems. 
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• The representation of numbers, as physically measured on the surface of the 
graphic itself, should be directly proportional to the numerical quantities 
represented. 

• Clear, detailed, and thorough labeling should be use to defeat graphical 
distortion and ambiguity. Write out explanations of the data on the graphic 
itself Label important events in the data. 

Tufte theorizes that in order to get an idea of how much a graphic is in 
violation of the first principle, one could measure it by use of the “Lie Factor” which is 
the size of effect shown in the graphic divided by the size of effect in data [=(size of 
graphic)/(size of data)]. The Lie Factor is used to exaggerate differences or similarities. If 
the Lie Factor is equal to one, then the graphic is probably doing an adequate job of 
accurately representing the numbers. If the Lie Factor is greater than 1.05 or less than .95, 
then substantial distortion is possibly involved. As an example, Tufte uses a newspaper 
that reported that the U.S. Congress and the Department of Transportation had set a series 
of fuel economy standards for automobile manufactures, beginning with 18 miles per 
gallon in 1978 and moving incrementally up to 27.5 miles per gallon by 1985, an increase 
of 53 percent. However, when the magnitude of the change from 1978 to 1985 is shown 
in the graph by the relative lengths of the two lines, it reveals a difference of 783 percent. 
Thus, the numerical change of 53 percent is presented by two lines that changed 783 
percent, yielding a Lie Factor of 14.8 percent, well beyond the bounds of acceptability. 

The Lie Factor could be a quick and easy method for gaining insight into 
the scale of the graphics used to depict candidates’ PSR information to the board. For 
example, suppose a candidate had a trait average of 4.80 for a reporting period and during 
the subsequent period, he had a report average 4.60 for a negative .20 difference which is 
a 4.16 percent decrease of the first mark of 4.80. If the graph depicting that difference 
changed from 1.25 inches to LOO inch, we calculate a decrease of a 20 percent for a Lie 
Factor of (.20/.0416 = 4.81) which, in accordance with Tufte’s advice is substantially 
distorted because it is greater than the 1.05 upper limit that he recommends. In this case 
you would most likely increase the scale of the graph (otherwise the difference may not 
be visible to the viewers) which would have the effect of decreasing the difference or 
variation being shown. 
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The accurate depiction of data also means that design variation should not 
affect or distort the data being displayed. In other words, one should not change the data 
in order to make the graphic look nice. The compounding of design variation with data 
variation over the surface of a graphic leads to ambiguity and deception, because the eye 
may mix up changes in the design with changes in the data. The graphic should follow 
the principle of showing data variation, not design variation. 

Another way to confuse data variation with design variation is to use the 
space to show one-dimensional data. A common mistake is to use areas to show 
magnitudes and falsely varying both dimensions simultaneously in response to changes in 
one-dimensional data. Vast ambiguities arise when viewers perceive a two-dimensional 
surface and then convert that perception into a non-dimensional number. The difference 
in physical area on the surface of a graphic does not reliably produce appropriately 
proportional changes in perceived areas. This problem is even worse when adding a third 
dimension to the graphic. The use of two or three varying dimensions to show one 
dimensional data is not an efficient technique and often leads to ambiguity in perception 
and should be avoided by abiding by the following principle: The number of 
information-carrying (variable) dimensions depicted should not exceed the number of 
dimensions in the data. 

Tufte writes that to be truthful and revealing, data graphics must bear on 
the question at the heart of quantitative thinking: “Compared to what?” The emaciated, 
data-thin design should always provoke suspicion, for graphics often lie by omission, 
leaving out data sufficient for comparisons. The principle: Graphics must not quote data 
out of context. 

On the issue of Graphical Integrity, Tufte points out that graphical 
proficiency requires three entirely different skills: the substantive, statistical, and artistic. 
If statistical integrity and graphical sophistication are to be achieved, then the team of 
designers should include experts in the area of substantive and quantitative information. 
Don’t leave the design work solely up to the graphic artists or you may end up with 
graphics that (1) lie; (2) employ only simple designs; (3) and do not capture the real news 
of the data. 


28 



To summarize, in order to maintain graphical integrity it is important for 
the designer to ask the right questions: 

• Does the display tell the truth? 

• Is the representation accurate? 

• Are the data documented? 

• Do the display methods tell the truth? 

• Are the appropriate comparisons, contrasts, and contexts shown? 

2, Theory of Data Graphics 

a. Data-Ink and Graphical Redesign 

Tufte’s fundamental principle of good statistical graphics is: “Above all 
else show the data.” This principle is the basis for a theory of data graphics. The majority 
of ink on a graphic should represent data-information, with the ink changing as the data 
changes. Tufte explains that data-ink is the non-erasable core of a graphic, the non- 
redundant ink arranged in response to variation in the numbers represented and should be 
maximized at all times. The more of the graphic that is devoted to data, the better off it 
will be in displaying that data. 

In conjunction with maximizing the data to ink ratio, he also writes that 
one should erase the data ink that is not representing data. Ink that doesn’t depict 
statistical information doesn’t have much use to the viewer of the graphic and could 
possibly cover up or clutter up useful data. One way to reduce unnecessary data ink is to 
look at the bilateral symmetry. Is there a chance that the space consumed in a graphic can 
be halved without losing any important information? Unless it serves a distinct purpose, 
redundancy of the same data should be avoided. It is important to be able to edit the data 
after it has been displayed in order to weed out the unnecessary or redundant data-ink so 
that the critical data is obviously clear. 

These five principles in the theory of data graphics produce substantial 
changes in graphical design. These principles apply to many graphics and yield a series 
of design options through cycles of graphical revision and editing. The five principles 
are: 
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• Above all else show the data 

• Maximize the data-ink ratio 

• Erase non-data ink 

• Erase redundant data-ink 

• Revise and edit. 

b. Chart Junk: Vibrations, Grids, and Ducks 

Throughout his books, Tufte mentions how to aehieve graphieal 
excellence by avoiding many graphical mistakes or methods . 

Sometimes a designer of a graphic unintentionally engrosses the body of 
the graphic with a lot of ink that serves no useful purpose and doesn’t tell the viewer 
anything new. The purpose of superfluous decoration may have been to make the graphic 
appear more scientific, to enliven the display, or to give the designer an opportunity to 
exercise artistic skills. Whatever the reason, the excess is all non-data-ink or redundant 
data-ink, and is what Tufte refers to as chart junk. Chart junk consists of decorative 
elements that provide no data and cause confusion. This graphical decoration is prevalent 
in technical publications as well as in commercial and media graphics and is easier to 
produce than the hard work required to produce intriguing numbers and securing 
evidence. Most chart junk doesn’t involve artistic considerations or motivations, rather it 
just naturally evolves and pervades by use of conventional graphical bits and pieces that 
are routinely used in displays. These bits and pieces consist of over-busy grid lines and 
excess ticks, redundant representations of the simplest data, the debris of computer 
plotting, and many of the devices generating design variation. 

Tufte points out three widespread types of chart junk that are typically 
abundant in scientific and technical research these days. They are the unintentional 
optical art, the dreaded grid, and the graphical duck. 

Unintentional optical art is dependent upon moire effects, in which the 
design interacts with the physiological quiver of the eye to produce the disturbing 
manifestation of vibration and movement. This disturbing effect extends beyond the ink 
of the design to the whole page. It is not a good idea to draw up statistical graphics that 
appear to shimmer. As bad as this moire vibration is in clouding the flow of information, 
it is even more abundant today due to the ability to easily and quickly create the ill effects 
with computer programs such as Excel and Power Point. “The handbooks and textbooks 


30 



of statistical graphics, along with the user’s manuals for eomputer graphies programs, are 
filled up with vibrating graphies, presented as exemplars of design” (Tufte, 2001). 


Another form of ehart junk is that of the grid. If the designer should 
attempt to use any form of grid it should be very light or eompletely suppressed showing 
only an implieit presenee so as not to interfere with the target data. Grids are used for the 
initial plotting and should not show up on the final graphie. Grids eontain no information, 
tend to elutter up the graphie, and generate graphie aetivity unrelated to data information. 

When a graphie is taken over by deeorative forms or eomputer debris, the 
data measures and struetures beeome design elements, or the overall design purveys 
graphieal style rather than quantitative information, then that graphie may be ealled a 
duck in honor of the duek-form store, “Big Duck.” For this building the whole strueture 
is itself decoration, just as in the duek data graphie. (Tufte, 2001) 

Designers of data graphies should seek to use new teehnology in a way 
that present data in a meaningful way rather than quickly and massively spitting out 
graphies that just look eool. Chart junk can turn a mediocre graphie or chart into a total 
disaster, but it ean never replaee or effeetively represent a thin data set. A designer should 
never use ehart junk, ineluding moire vibration, the grid, or the duck. 

c. Data-Ink Maximization and Graphical Design 
By using the previous prineiples, it is possible to not only edit old designs 
but derive new graphieal forms as well. Tufte uses several examples where he shows 
before and after effeets of simple graphs to show that his theory of data graphies is 
eorreet. (See Figure 5). In some of his examples, the observed inereases in effieiency, or 
how much of the graphie’s ink carries information, were sometimes quite large. In 
several cases, the data-ink ratio inereased from .1 or .2 to nearly 1.0. These transformed 
designs were less eluttered and eould be shrunk down more easily than the originals. But 
are these transformed ideas better? Tufte answers this question by saying: 
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• They are better within the prineiples of the theory, for more information per 
unit of space and per unit of ink are displayed. It’s a significant gain when you 
consider that the history of devices for communicating information is written 
in terms of increases in efficiency of communication and production. 

• Graphics are almost always going to improve as they go through editing, 
revision, and testing against different design options. The principles of 
maximizing data-ink and erasing generate graphical alternatives and also 
suggest a direction in which revisions should move. 

• Then there is the audience: will those looking at the new designs be confused? 
Some of the designs are self-explanatory. The chance of a member of the 
audience being confused is remote. Nothing is lost to those who are puzzled 
by the changes and something is gained by those who do understand it. 
Moreover, it is a frequent mistake in thinking about statistical graphics to 
underestimate the audience. 

• Some of the new designs may appear odd, but this is probably because we have 
not seen them before. The conventional designs for statistical graphics that 
have been viewed thousands of times where as the new designs have not. In 
time and with use the new designs would become just as reasonable as the old. 
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Figure 5. Before and after versions of numerical charts. (From Salon) 

d. Multi-functioning Graphical Elements 

The same ink should often serve more than one graphical purpose at a 
time. A graphical element may carry data information and also perform a design function 
usually left to non-data-ink. Or it might show several different pieces of data. Such multi¬ 
functioning graphical elements, if designed with care and subtlety, can effectively display 
complex, multivariate data. (Tufte. 2001) The principle is: Mobilize every graphical 
element, perhaps several times over, to show the data. 

In other words, use the graphic segment to show or portray more than one 
item of data. Graphical structures can often serve several purposes at once. For example, 
the stem and leaf plots display sequences of numbers which directly portray structure by 
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the physical length of each sequence. Tufte contends that the designers of such graphical 
elements must be careful because multi-functioning elements tend to generate graphical 
puzzles, with encoding that only the designer can figure out. The designers must strive 
for enhancing graphical clarity in the face of complexity when developing multi¬ 
functioning elements. 

Those graphical elements that actually locate or plot data in a graph or 
chart are called the data measure. For example, the bars of a bar-chart, the dots of a 
scatter-plot, the dots and dashes of a dot-dash-plot, or the blots of a blot map are data 
measures. The data grid itself may be the data, revealing both the values and the 
coordinate system at the same time. The ink that makes up the data measure can itself 
carry data, for example, the dots of the scatter-plot can take on different shading in 
response to a third variable. This rule can also be used with data-based coordinate lines as 
well. Why not use that ink to show data in the form of data-based labels? 

In many graphics the coordinate labels are separated from the data 
measures. Because of that the eye of the viewer must move back and forth between the 
path formed by the data and the coordinate positions arrayed along the margins of the 
graphic. It is possible that this eye work could be eliminated entirely by turning the 
coordinate labels into data measures, another double functioning maneuver. 

e. Colors 

Much of Tufte’s books talk about the effect that colors have on graphics. 
However, much of what he writes about does not apply directly towards the graphics that 
we are concerned with in this study so the we have tried to compile only the relevant 
applications of using color as they may apply in the redesign of the promotion displays. 

Of course color brings to information more than just codes naming visual 
nouns - color is a natural quantifier, with a perceptually continuous (in 
value and saturation) span of incredible fineness of distinction, at a 
precision comparable to most measurement (Tufte, 1990). 

Color use can greatly enhance data comprehension and can convey multi¬ 
dimensional values effectively. Color grids are a form of layering which provides context 
but which should be unobtrusive and muted. Pure bright colors should be reserved for 
small highlight areas and almost never used as backgrounds. Colors can be use as labels. 
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as measures, and to imitate reality as in the blue lakes in maps. Color spots against a light 
gray are also effective. 

Despite these benefits of using colors in displaying information, the 
designer must avoid using colors in a way that generates graphical puzzles. Despite our 
experiences with the spectrum in science textbooks and rainbows, the mind’s eye does 
not readily give a visual ordering to colors, except possibly for red to reflect higher levels 
than other colors (Tufte 2001). Attempts to give colors an order often requires the viewer 
to mentally decode the color scheme, trying to recall what the different colors 
represented, and to look back at the legend to clarify it. 

The success of gray compared to the visually more spectacular color gives 
us a lead on how multi-functioning graphical elements can communicate complex 
information without turning into puzzles. The shades of gray provide an easily 
comprehended order to the data measures. This is the key. Central to maintaining clarity 
in the face of the complex are graphical methods that organize and order the flow of 
graphical information presented to the eye. 

So what colors should we use in order to represent and illuminate 
information? Tufte suggests the use of colors found in nature, especially those on the 
lighter side, such as blues, yellows, and grays of sky and shadow. 

Nature’s colors are familiar and coherent, possessing a widely accepted 
harmony to the human eye - and their source has a certain definitive 
authority. A palette of nature’s colors helps suppress production of garish 
and content-empty color junk. Local emphasis for data is then given by 
means of spot highlights of strong color woven through the serene 
background (Tufte, 1990). 

In their book Visualization: Using Computer Graphics to Explore Data 
and Present Information, the authors explain that in displays of graphs and charts, there 
are overlapping lines, grids, background, and text. Thus a designer needs to select colors 
so that the various pieces of information are distinguishable even when they are 
overlapping. “The connotation of color is critical in monitoring applications because the 
viewer must react quickly to the appearance of a particular color in a given location.” 
(Brown, et al. 1995) Table 3 and Table 4 reproduced from their book show the best and 
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worst color combinations, as determined from a study where at least four of sixteen 
partieipants aeeepted or rejeeted eaeh eolor eombination. 


Best Color Combinations 
(N = 16) 


Background 

Thin Lines and Text 

Thick Lines and Panels 

White 

Blue (94%), Black (63%), Red (25%) 

Black (69%), Blue (63%), Red (31%) 

Black 

White (75%), Yellow (63%) 

YeUow (69%), White (50%), 

Green (25%) 

Red 

Yellow (75%), White (56%), Black (44%) 

Black (50%), YeUow (44%), 

White (44%), Cyan (31%) 

Green 

Black (100%), Blue (56%), Red (25%) 

Black (69%), Red (63%), Blue (31%) 

Blue 

White (81%), Yellow (50%), Cyan (25%) 

Yellow (38%), Magenta (38%), 

Black (31%), Cyan (31%), White (25%) 

Cyan 

Blue (69%), Black (56%), Red (37%) 

Red (56%), Blue (50%), Black (44%), 
Magenta (25%) 

Magenta 

Black (63%), White (56%), Blue (44%) 

Blue (50%), Black (44%), YeUow (25%) 

Yellow 

Red (63%), Blue (63%), Black (56%) 

Red (75%), Blue (63%), Black (50%) 

Table 3 

Best Color Combination. (From Brown, et al.) 

Worst Color Combinations 
(N= 16) 

Background 

Thin Lines and Text 

Thick Lines and Panels 

White 

YeUow (100%), Cyan (94%) 

YeUow (94%), Cyan (75%) 

Black 

Blue (87%), Red (37%), Magenta (25%) 

Blue (81%), Magenta (31%) 

Red 

Magenta (81%), Blue (44%), 

Green (25%), Cyan (25%) 

Magenta (69%), Blue (50%), 

Green (37%), Cyan (25%) 

Green 

Cyan (81%), Magenta (50%), 

YeUow (37%) 

Cyan (81%), Magenta (44%), 

YeUow (44%) 

Blue 

Green (62%), Red (37%), Black (37%) 

Green (44%), Red (31%), Black (31%) 

Cyan 

Green (81%), YeUow (75%), White (31%) 

YeUow (69%), Green (62%), 

White (56%) 

Magenta 

Green (75%), Red (56%), Cyan (44%) 

Cyan (81%), Green (69%), Red (44%) 

Yellow 

White (81%), Cyan (81%) 

White (81%), Cyan (56%), Green (25%) 

Table 4 

Worst Color Combinations. (From Brown, et al.) 


f. Data Density and Small Multiples 

Graphies are best when they represent very dense and rieh data sets. 
Within a small area, our eyes ean make an extraordinary number of distinctions. For 
example, by using very light grid lines it is easy to locate 625 points in one square ineh. 
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or equivalently, 100 points in one square eentimeter. It would be wise then in the design 
of statistieal graphies to take advantage of the eye’s ability to deteet large amounts of 
information in small spaees. But how does one gauge how mueh information the graphies 
should show? Tufte ealls this the measure of data density. Taking into aeeount the size of 
the graphie in relation to the amount of data it displays yields the data density: 

Data density = (no. of entries in data matrix)/(area of graphie) 

Figure 6 of the New York City weather history is a good example of a 
graphie with a high data density of 181 numbers per square ineh (28 per square 
eentimeter). 



Figure 6. New York Weather History (From Tufte) 


Data graphies should be based on large rather than small data matriees and 
have a high rather than low data density. More information is better than less information, 
espeeially when the marginal eosts of using and interpreting additional information are 
small, as they are for most graphies. Tufte also reeommends that the simple items should 
left out in tables or in the text. Graphies ean give a sense of large and eomplex data sets 
that eannot be managed in any other way. If it beeomes overerowded (although several 
thousand numbers represented isn’t usually a problem) a variety of data reduetion 
teehniques - averaging, elustering, smoothing - ean thin the numbers out before plotting. 
It is important to realize that summary graphies ean emerge from high-information 
displays, but there is nowhere to go if we begin with a low-information design. 

Graphieal designs that are data- rieh give eontext and eredibility to 
statistieal evidenee. Viewers may question low-information designs by wondering what is 
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left out, what is hidden, and why are we shown so little. Graphies that are high density 
will help the viewer to compare parts of the data by displaying much information within 
the view of the eye. The principle to remember here is: Maximize data density and the 
size of the data matrix, within reason. 

One must be careful when designing high- information graphics because 
as the volume of data increases, data measures must shrink (smaller dots for scatter plots 
or thinner lines for time series, etc.) The appearance and disorder of chart junk, non-data- 
ink, and redundant data-ink can result in even greater disaster in data-rich designs. The 
way to increase data density without having to enlarge the data matrix is to reduce the 
area of the graphic. Tufte points out that many data graphics can be reduced in area to 
half their currently published size with virtually no loss in legibility and information. 

In review, good graphics are: 

• Comparative 

• Multivariate 

• High density 

• Able to reveal interactions, comparisons, etc. 

• Drawn almost entirely with data-ink 

g. Aesthetics 

Tufte writes that graphical elegance consists of design simplicity , 
complexity, and truth of data. Graphics that are visually striking generate power from 
content and interpretations beyond the immediate display of some numbers. To attain 
attractive displays of statistical information a graphic should: 

• Have a properly chosen format and design . 

• Use words, numbers, and drawing together. 

• Reflect a balance, a proportion, and a sense of relevant scale. 

• Display an accessible complexity of detail. 

• Often have a narrative quality; a story to tell about the data. 

• Be drawn in a professional manner, with the technical details if production 
done with care. 

• Avoid content-free decoration, including chart junk. 

Tufte explains that words and pictures belong together and sometimes 
viewers need help that words can provide. It is almost always helpful to write little 
messages on the plotting field to explain the data, to label the outliers and interesting data 
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points, to write equations and sometimes tables on the graphic itself, and to integrate the 
captions and legend into the design so that the eye is not required to dart back and forth 
between textual material and the graphic. The size of type on and around graphics can be 
quite small, since the phrases and sentences are usually not too long - and therefore the 
small type will not fatigue viewers the way it does in lengthy texts. The principle of 
data/text integration is: Data graphics are paragraphs about data and should be treated as 
such. 

When graphical elements are together and their proportions are in balance, 
it looks better and ensures that an integrated quality and appropriate visual linkage exists 
between them. Lines in data graphics should be thin. The orthogonal intersection of lines 
of different weights is an effective aesthetic device as well. 

The design of data graphics should tend toward the horizontal rather than 
the vertical. Tufte believes that a display greater in length than height has advantages due 
to the fact that our eye is naturally practiced in detecting deviations from the horizon, and 
graphic design should take advantage of this fact. Horizontally stretched time series are 
more accessible to the eye. The analogy to the horizon also suggests that a shaded, high 
contrast display might occasionally be better that the non-shaded lines that resemble a 
floating snake. It is also easier to read words that read from left to right on a horizontally 
stretched plotting field. A longer horizontal axis helps to elaborate the workings of the 
causal variable in more detail. How much wider should it be? Tufte writes that a good 
design would move toward the horizontal about 50 percent wider than tall. 

3. Conclusion 

The importance of a well thought-out, and coherent design of the graphics for the 
promotion visual displays cannot be overstated. The rapid and accurate transfer of 
information by use of sound graphical principles can have a profound effect on future of 
the Navy’s promotion system. In determining the design of choice, the principles set forth 
above should help generate design options and guide choices among options. It is Tufte’s 
belief that these principles should be applied flexibly because they are not logically or 
mathematically certain and it would be better to violate any principle than to place 
graceless or inelegant marks on paper. 
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What is to be sought in design for the display of information is the elear 
portrayal of eomplexity. Not the eomplieation of the simple; rather the 
task of the designer is to give visual access to the subtle and the difficult - 
that is, the revelation of the complex (Tufte, 2001). 
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IV. DATA COLLECTION AND ANALYSIS 


A. INTRODUCTION 

The proof-of-concept (POC) framework for this study is the Navy’s current 
officer promotion/selection process. In order to answer the principle research question we 
have developed a POC prototype visualization solution through the use of COTS 
software and technology in order to display, at a minimum, visualization concepts and 
capabilities currently available. This prototype uses a simple notional relational database 
(Microsoft Access) and simulates the current system’s input and outputs. Using three-tier 
architecture, the prototype will extract the data from the database and display an 
improved visual display for the board to interpret and compare. Additionally, because the 
system is web- based, this data would also be available to the individual briefer in order 
to prepare his or her records for the tank briefings. We propose that by utilizing a web- 
based system such as our POC and allowing electronic documents to be produced and 
shared via the network the new system will substantially reduce the time and increase the 
effectiveness of the promotion process. In order to provide evidence of this, we have 
created the following sections: 

• An “as-is” analysis with a Return on Knowledge (ROK) spreadsheet 
and graphical representation. 

• A “to-be” incremental model spreadsheet. 

• Comparison of the “as is” and “to be” models with KVA analysis. 

• A fully functional web site that utilizes a notional relational database. 

• Prototype web-based charts and graphs using different colors, along 
with layering and separation multidimensional schemes that would 
facilitate information flow. 

The focus of the chapter will be on the Knowledge Value Added methodology 
which will provide us with a performance ratio for all the core processes involved in 
order to determine if there is any increase in effectiveness between the “as-is” and “to- 
be” models. 

B, DATA COLLECTION 

Our data collection process focused on obtaining the most suitable information 
that would provide us with the necessary background to answer the research questions. 
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Here we will explain how we colleeted the data. The knowledge assessment - associated 
with the processes identified during our data collection will identify opportunities for 
increased returns. The scope of our data collection was limited to the officer promotion 
process starting from the pre-board stage, beginning with uploading the list of eligible 
officers, up until the final Tentatively Selected (TS) list is produced. By following the 
seven steps of the KVA process as listed in Table 1 of chapter 3, we were able to 
calculate the ROK for the “as is” promotion process. 

1, Step One: Identify Core Process and Sub-processes 

Our first step was to identify the core process and its sub-processes. In order to 
establish the boundaries of the core process we needed to describe in detail the “as is” 
process: 

• Identify the end output of the core process. 

• Identify the sub-process outputs that eventually create the end product. 

• Create and aggregate-level process diagram that depicts the inputs and outputs 
of the core and each of its sub-processes. Figure 7 is a depiction of our 
workflow model. 

The information that was collected on the “as is” process was gathered through 
interviews with the subject matter experts (SMEs) involved in the overall process. The 
SMEs were asked to break the overall process down into smaller, more specific sub¬ 
processes through which we could obtain average learning-time estimates and the number 
of roughly equivalent process instructions required to complete each sub-process, from 
this data we were able to break up the core promotion process into the following five 
separate and distinct processes in which to base our “as is” base line model: 

• IT Review 

• Assistant Recorder Review 

• Briefer Review/Annotation 

• Data Downloads 

• Brief/Presentation/V ote 
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Figure 7. “As Is” Process Model 

a. IT Review 

The first step in the typical promotion board process starts about four 
months before the convening date. This is when the initial list of eligible officers is 
compiled and modified as required. The eligible list is continually synchronized with an 
official automated database to ensure consideration of all candidates. The master file is 
queried six weeks prior to the board convening for FITREP continuity and messages are 
sent for those that are missing. The specific process instructions were identified as: 
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• Upload list of eligible. 

• Perform FITREP continuity check. 

• Create a discrepancy database of record with gaps greater than 90 days. 

• Post list of web for Fleet access. 

• Obtain UICs for command’s members missing FITREPS. 

• Send messages to Fleet notifying members of gaps in records. 

• Eog all faxed FITREPS received and check for validity. 

• Update website list of missing FITREPS. 

• If it’s a valid FITREP, then send to the system scanner; if not a valid FITREP, 
then notify command. 

• Prepare board room and tank for use. 

• Provide board members and recorders with welcome aboard packages. 

b. Assistant Recorder Review 

For most boards the recorders arrive one week prior to the board actually 
being convened. During this week, the assistant recorders are trained in the board's 
administrative procedures and assigned approximately 80 service records to review. Once 
proficient, a service record review takes 15 to 20 minutes. 

This is the first time that someone other than the eligible will review the 
record as it will appear before the selection board. The record is reviewed for several 
items including regular report continuity for the last five years. The OSR/PSR is checked 
to see that FITREP grades are accurate and the record is reviewed to ensure that award 
citations are present for the number and type of awards that are listed on the OSR/PSR. 
The assistant recorder also verifies that an eligible has a photo in current grade. 

Any documentation the board receives, either directly from the service 
member or through the detailer, is logged into the system and scanned into the selection 
board copy of the service record. Only then can the assistant recorder make the necessary 
changes on the appropriate Summary Cover Sheet and the PSR. The process instructions 
involved include: 
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• Electronically select and open record for review. 

• Check that the number of awards in written comments is the same in OSR/PSR 
page. 

• Check that the education in written comments is the same as on the OSR/PSR. 

• Check that the grades on FITREPS are the same as on the OSR/PSR. 

• Check that photo is current. 

• Prepare a list of discrepancies. 

• Send messages to member’s commands with missing data. 

• Accept and enter late data. 

• Send to the board members administrative requirements. 

c. Briefer Review/Annotation 

This sub-process begins once the actual board convenes. Basically, once 
the board members are checked in, they will receive the administrative and welcome 
aboard briefs. They will then be briefed on the board precept; a document that is signed 
by the Secretary of the Navy and is delivered to the board president. It gives guidance on 
how to conduct the board and is written each time there is a promotion board. The 
members will then be given a number of records to review, annotate, and grade in 
preparation for the presentation to the rest of the board. The process instructions include: 

• Review Precept letter: 

- selection standard 

- authorized percentage to select 

- cause determination 

- general procedures and schedule 

- equal opportunity guidelines 

- results management guidelines 

• Select records for review 

• Review and annotate exceptional performance in operational environment, 
command leadership, integrity, resourcefulness, and innovation. 

• Review and annotate graduate education/specialty training. 

• Review and annotate clinical proficiency/skill, management skills. 

• Review and annotate innovation and retention effectiveness. 

• Review and annotate antiterrorism/force protection qualifications. 

d. Data Download 

After the individual records have been reviewed, annotated, and graded, 
the reports are then saved to a disk and then separately downloaded into the selection 
board voting tank module. This procedure often results in a substantial slow down in the 
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overall process because the information is not immediately available to the voting 
members in the tank. For example, once enough records have been reviewed and a ta nk 
is available (often there is more than one board in session so the tanks may be 
unavailable) the senior member may decide to go to the tank for the vote. This may be 
delayed because the reviewed records now have to be separately downloaded into the 
voting tank module. The sub-process includes: 

• Save annotated data file to disk. 

• Forward data file for manual download to the board voting tank. 

e. Brief/Presentation/Vote 

After the records have been downloaded into the tank, all of the voting 
members gather in the tank where the member who reviewed the record briefs it while 
the eligible individual’s OSR/PSR and cover page are displayed and all board members 
vote. Sub-process: 

• Briefer displays and orally reviews annotated data. 

• All members compare with experience baseline for: 

- Exceptional performance in operational environment, command 
leadership, integrity, resourcefulness, and innovation 

- Graduate education/specialty training 

- Clinical proficiency/skill, management skills 

- Innovation and retention effectiveness 

- Anti-terrorism/force protection qualifications 

• All members cognitively “normalize” graded averages and the R.S.’s 
cumulative average. 

• All members cast vote for each record, cognitively normalizing all data with 
regard to all eligible records. 

2. Step two: Establish Common Units to Measure Learning Time 

In step two, we were able to establish, through our interviews with the 
SMEs, a common definition of learning time (ET) for the five processes under review. 
ET is the amount of time it takes for an average person to learn how to complete the 
process correctly and is directly correlated to the amount of knowledge embedded in a 
given process. 

3, Step three: Calculate Learning Time to Execute each Suh-process 

With the SMEs input, we were able to determine the total time it would take to 


learn how to execute each sub-process. Given that there were an estimated total of 100 
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weeks to learn how to execute the five processes; the distribution of learning time is 
depicted in Table 5 below. 


Process: 

Learning time 
(months) 

Data Download 

1 

IT Review 

6 

Assistant Recorder Review 

8 

Briefer Review/Annotation 

41 

Brief/Presentation/V ote 

44 

Total 

100 


Table 5 Actual Learning Times 


4. Step four: Designate Sampling Time Period 

In step four, instead of designating a sampling time period in order to capture a 
representative sample of the compound processes’ final output, we used the number of 
employees within each core area to determine the weighted amount of knowledge 
executed in the process per 1000 records. The “Persons involved” category reflects the 
number of people involved in completing a process. Accounting for the number of 
employees gives a general idea of how often knowledge (K) is fired and provides a 


rough-cut way of weighting knowledge in the processes over the evaluation period 
(Housel and Bell, 2001). See Table 6. 



Value added 
(hours to train) 

Persons involved 
(weighting factor) 

Time 

subtotal 

Data Download 

.2 

1 

.2 

IT Review 

.5 

3 

1.5 

Assistant Recorder Review 

.8 

7 

5.6 

Briefer Review/Annotation 

3 

1 

3 

Brief /presentation/Vote 

5 

11 

55 

Totals: 

9.5 

23 

65.3 


Table 6 Step 4: Weighting factor 


5, Step five: Multiply the Learning Time for each Suh-Process 

In this case we took the time subtotal and multiplied it by the number of times the 

knowledge for that particular sub-process was fired or actually used. The number of 

times fired is a close approximation from the SME in that field of how many times per 

1000 records that the requisite knowledge for that process was actually used. We then 
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used that subtotal and multiplied it by the estimated number of iterations of the sub- 
proeess to be executed during a board for each 1000 records. The result is the weighted 
value which is ultimately used as the top half of the return ratio. In other words, the sum 
of the total amount of learning time knowledge for each process is used as the numerator 


in the ROK equation. It is the knowledge required to get the outputs. See Table 7 below. 



Time 

Subtotal 

X#of 

times 

fired 

Fired 

subtotal 

X#of 

iterations 

= Weighted 
Value Added 

Data Download 

.2 

1 

.2 

6 

1.2 

IT Review 

1.5 

1 

1.5 

1 

1.5 

Assistant/Recorder 

5.6 

143 

800.8 

1 

800.8 

Brief Review 

3 

83 

249 

2 

498 

Brief/Zpresentation 

55 

1000 

55000 

2 

111000 

Total: 

65.3 




111301.5 


Table 7 Step 5 of the KVA Calculation 


6, Step 6: Allocate Revenue 

In step six we calculate the cost to execute each sub-process based solely on the 
cost of the personnel at each sub-process. Rather than trying to figure out the different 
TAD (Temporary Additional Duty) costs for each rank, which is too complex, we chose 
to normalize the cost data in the following way. If the process used two E-5 personnel 
then we estimated how long in hours it took to complete the process step and then 
determined the cost based on the average time of service of a typical E-5 in the Navy. 
Eor example, an E-5 over six years of service gets paid X dollars per month divided by 80 
hours per month. We then determined that if the process step took eight man-hours then 
eight man-hours times the hourly rate equals the total sub-process cost. Here we 
attempted to isolate just those costs that pertain to the creation of the process outputs. 
The estimates were based on the SMEs approximations on how long it took individuals to 
generate the outputs necessary to complete a specific task. Eor example, referring to table 
XXX, we estimated that for every 1000 records reviewed, six separate data downloads 
were required on average, based on 160 records reviewed before being sent to the tank. 
We estimated that each download would take about 1 hour for a total execution time for 
that process of six hours per 1000 records reviewed. See Table 8. 
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Execution time (hours) 

Process Costs ($) 

Data Download 

6 

$40.70 

IT Review 

72 

$366.30 

Assistant/Recorder 

280 

$1852.80 

Brief Review 

360 

$27,633.59 

Brief/Zpresentation 

120 

$29,655.56 

Total; 


$59,548.95 


Table 8 Step 6 of the KVA Calculation 


7. Step 7: Calculate ROK and Interpret the Results 

In the last step we computed the ROK for each sub-process by taking the 
weighted value added for each sub-process and dividing it by the process costs. By 
breaking the ROK figures down to the individual processes, we are then able to generate 
an order of magnitude estimate of the return, based solely on the ratio of cost to amount 
of knowledge and process instructions executed during the sample period. The overall 
ROK for all of the processes in the “as is” model is 186.91%. See Table 9. 


= Wei<^ited 

Piocess 

Execiiioii tinie 

Piocess Costs 

ROK"* 

/.line A<l(le<l 

liisliiictioiisUivwtl 

fhourst 

m 

C%) 






1.2 

2 

6 

$40.70 

2.95% 

1.5 

11 

72 

$366.30 

0.41% 

800.8 

9 

280 

$1,852.80 

43.22% 

498 

13 

360 

$27,633.59 

1.80% 

110000 

48 

120 

$29,655.56 

370.93% 

111301.5 

83 


$99,548.95 | 

186.91% 







Table 9 ROK Calculations 
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9 
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3 

1 
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Table 10 Overall “as is” KVA Analysis 


C. “TO BE” INCREMENTAL DESIGN KVA ANALYSIS 

With the base-line model we ereated, we produced our incremental “to-be” model 
and compared the two for any differences in the ROK figures. Additionally, this base line 
model can be used to compare any future re-design efforts, including a radical redesign of 
the process such as using the five vector model for the purposes of promotions. As 
explained previously, our “to-be” incremental design is envisioned to be web-based and 
thus introducing advanced automation and accessibility options into the process. A major 
benefit we foresee is eliminating redundant or repetitive sub-processes such as the data 
download stage prior to each tank session. Figure 8 shows the redesigned or incremental 
process model. 

There is one major difference between the “to-be” and “as-is” model. In the “to- 
be” model, we completely eliminated the sub-process for the data being manually 
downloaded to the tank. Although this has only really eliminated two process 
instructions, we show in our spreadsheet analysis that this equates to a significant time 
reduction in the overall process. 
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The following is a list of the “to-be” sub-proeesses with their respective process 
instructions: 

• IT Review 

- Process instructions remain the same as the “to-be” model. 

• Assistant Recorder Review 

- Process instructions remain the same 

• Briefer Data Review and Formatted for Presentation 

- Process instructions remain the same 

• Brief PresentationA^ote 

- Briefer displays and orally reviews annotated data. 

- All members visually compare record with peer performance aggregate 
baseline data. 

- All members cast vote for each record. 


For the scope of our POC model, the sub-processes and process instructions for 
the incremental design are essentially the same as the “as-is” model until after the Briefer 
Review/Annotation sub-process. However, with the current work being done through the 
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Navy’s personnel and manpower modernization effort, it is possible for all of the sub¬ 
process to be significantly reduced, thus allowing for much higher ROK figures in the 
future. For the scope of this project however, we have decided to limit the re-design to 
only those sub-processes that would be directly affected by our POC model. The 
following table (Table 11) lists the specific numerical difference in the ROK analysis for 


the “to-be” model: 
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Table 11 Overall “to-be” KVA analysis 


D. COMPARATIVE ANALYSIS 

The KVA methodology offers a means in which to objectively measure the value 
of knowledge assets deployed in core processes. The value measurement of knowledge 
embedded in an organization’s core processes, technology, and employees is 
accomplished by determining what the organization’s Return on Knowledge (ROK) is. 
The numerator of the ratio represents the percentage of the revenue or sales dollar 
allocated to the amount of knowledge required to complete a given process successfully, 
in proportion to the total amount of knowledge required to generate the organization’s 
total outputs. The denominator of the ratio is the cost to execute the process knowledge. 
By comparing the “as-is” figure in Table 12 to the “to-be” model in Table 13, we see a 
significant difference in the ROK figures. The “to-be” model is a close approximation of 
the redesigned web-enabled promotion process to include the enhanced OSR/PSR 
graphical display. 
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2 

6 

$40.70 

2.95% 

11 

72 

$366.30 

0.41 % 

9 

280 

$1,852.80 

43.22% 

13 

360 

$27,633.59 

1.80% 

48 

120 

$29,655.56 

370.93% 

83 


$59,548.95 

186.91% 

Table 12 

“As-is” ROK figures 




Piocess 

Execution time 

Piocess Costs 

ROK*" 

Instiiictions (iiwt) 

(hours) 

__ 

_ m _ 


2 

6 

$40.68 

2.95% 

11 

72 

$366.48 

0.41 % 

9 

280 

$1,853.60 

43.20% 

13 

240 

$18,422.40 

2.70% 

48 

60 

$14,827.80 

741.85% 

83 


$35,510.96 

313.43% 


Table 13 “To-be” ROK figures 

As evident from the figures, the ROK does not signifieantly ehange until we 
ehange the later sub-proeesses of Briefer Review and the Brief/PresentationA^ote. In the 
last ease we ean see that with the new proeess, the ROK in last sub-proeess has doubled 
from 370.93 pereent to 741.85 pereent. In other words, the amount of knowledge 
exeeuted during that time frame (knowledge is a surrogate for the proeess outputs 
measured in eommon units) is signifieantly higher in the Brief/PresentationA^ote sub- 
proeess with the “to-be” ineremental design. 

Does the overall inerease in ROK from 186.91 percent to 313.43 percent support 
our principle research questions? 

• Can data-information visualization best practices be embedded in information 
technology in order to improve the overall effectiveness and objectiveness of 
the selection board process? 

• Will integrating web-based technology with the visualization techniques 
reduce the time required to prepare, brief, and make decisions in the promotion 
process? 
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These questions are really about improving the effectiveness and efficiency of the 
decision making process. Effectiveness in decision making is focused on what should be 
done, whereas efficiency is focused on how we should do it. George M. Marakas, in his 
book Decision Support Systems in the 2E‘ Century, explains the difference between the 
two in the context of how a decision support system contributes to each: 

Effectiveness 

• Easier access to relevant information. 

• Easter and more efficient problem recognition and identification. 

• Easier access to computing tools and proven model to compute choice criteria. 

• Greater ability to generate and evaluate large choice sets. 

Efficiency 

• Reduction in decision costs. 

• Reduction in decision time for same level of detail in the analysis. 

• Better quality in feedback supplied to the decision maker. 

By strictly comparing the KVA ROK figures of the before and after processes, we 
can easily see that the second question regarding efficiency has been improved. By using 
a fully web-enabled system, it would reduce process instructions involved in the later 
sub-processes which would result in an overall more efficient process. Eurther, we can 
surmise that by using our approach, the execution time in hours would decrease from 360 
hours in the Briefer Review/Annotation phase to 240 hours for every 1000 records. The 
big difference, however, was reducing execution time in the Brief/Presentation/Vote 
phase from 120 hours to 60 hours. This one improvement resulted in the dramatic 
increase in ROK. We believe that the data visualization techniques would help shorten 
the process instruction involved in the “voting” process due to its powerful ability to 
convert data objects into information so quickly. 

In the next chapter we will introduce our POC prototype web site that will shows 
how we can further increase the effectiveness by having; 

• easier access to relevant information 

• faster and more efficient problem recognition and identification 

• easier access to computing tools and proven models to compute choice criteria 

• provide a greater ability to generate and evaluate large choice sets 
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V. VISUAL ONLINE PROMOTIONS SYSTEM (VOPS) 


A. OVERVIEW 

In this chapter, we will explain the Visual Online Promotions System (VOPS), the 
system we developed to address the thesis objeetives. We begin by explaining the 
different seeurity approaehes we eonsidered and deseribe the eonstruetion of our 
information teehnology solution. The prototype helps the reader understand our redesign 
vision and supports the researeh questions of this thesis. 

B, SECURITY APPROACH 

The eurrent promotion system eonsists of the Eleetronie Military Personnel 
Reeords Management System (EMPRS) that eneompasses the Primary Input/Output 
system (PRIO), the Eitness Report/Evaluation system (EITEV), and Seleetion Board 
Module (SELBD). These modules work together to ereate a system that allows operators 
to manage member reeords from the desktop. All these systems are internal, meaning 
they ean only be aeeessed via the interfaees at the Navy Personnel Command in 
Millington Tennessee. The advantages to this approaeh inelude inereased seeurity and 
loeal administration of the intrieate information systems. 

The prototype we designed is intended to provide aeeess through the Internet, 
using eneryption and password proteeted aeeess. The advantages to having the system 
aeeessible from anywhere a person has Internet eonneetivity benefits not only the board 
members, who in the future will have the ability to peruse reeords from their own station, 
but also the promotion eandidate who ean aeeess the system and see what information 
about him or her is available, eurrent, and seen by the board. This web-enabled eoneept 
introduees another layer of seeurity eoneems and requirements beyond the seope of this 
thesis but we designed our system assuming these teehnieal and proeedural eoneerns will 
be researehed and implemented separately. 

The Visual Online Promotions System (VOPS) may be implemented using either 
a elosed intranet or publie Internet approaeh, but we designed our prototype to simulate 
aeeess via the Internet beeause we believe that future implementations will eventually use 
a web-enabled, virtual board system using some form of eollaborative teehnology. 
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C. WEB APPLICATION PROTOTYPE DEVELOPMENT 

To construct the VOPS, we used a variety of software eomponents. Speeifieally, 
Table 14 identifies and deseribes the purpose of eaeh software eomponent. 


Maker 

Software Title 

Version 

Purpose 

Maeromedia 

Dreamweaver MX 

6.0 

Design and draft static 
HTML pages and 
dynamie ASP pages 

Mierosoft 

Aeeess 

XP 

Create a database 
eontaining the 
promotion data 

Mierosoft 

Internet Explorer 

6.0 

View eompleted web 
pages and test 
applieation 

Mierosoft 

Internet 

Information Server 

6.0 

Loealized server 
software to enable 
Dreamweaver to display 
dynamie web pages 
(eonneets web page, 
database, and browser) 

Tal 

ble 14 Software solutions usee 

1 to develop VOPS prototy 


1. Overview of 3-Tier Architecture 

While the seope of this thesis is foeused on streamlining the Naval promotion 
system using a notional solution, the aetual solution will likely involve some form of 3- 
tier arehiteeture so we believe a eoneept overview is warranted. 

For web applieations sueh as what we are proposing, a 3-tier arehiteeture is 
eurrently the best solution. It eonsists of a elient tier, or presentation layer, where the user 
sees the information. The seeond tier is the applieation tier, or business-rule layer, and 
proteets the data from direet aeeess by the elients. The third tier, or data layer, is simply 
where the data is stored and is aeeessed via the applieation layer. The first two tiers are 
normally housed within the elient’s maehine with the data layer aeeessed through a 
network. 

Beeause the 3-tier arehiteeture proteets the data and provides a robust design, 
ereating the VOPS using this popular and powerful arehiteeture seheme provided the best 
solution for our design. 
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Using the software tools in Table 14, we set out to design the VOPS with a 
database, static web pages, and dynamic web pages using the Active Server Page (ASP) 
technology. By creating a notional Access database and designing web pages that 
dynamically query the data, we were able to implement a rapid prototype as a Proof Of 
Concept (POC). 

2, The Notional Database 

We designed a Microsoft Access database to house the VOPS data that we would 
query from the web pages. Figure 9 shows the schema we designed to normalize the data 
and to provide the necessary query capabilities to query a variety of data. 



Figure 9. Schema of the VOPS’s Notional Database 


Each block represents a table of information and the adjoining lines are 
relationships between the tables, hence a “relational” database. By isolating the data in 
this way, updates and maintenance can be accomplished cleanly without the problems 
associated with having redundant information in different locations. 
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It is important to note that the tables are never directly accessed but only via a set 
of queries designed to provide subsets of the data for the requested information, as seen 
in Figure 10. Note also that horizontal rows contain a record, or collection of data on one 
person. The vertical columns are the individual data fields for each record. Both queries 
and tables are organized in this manner. 



Figure 10. The Assignments Query 


We understand that Microsoft Access, while a powerful prototyping tool, is not 
scalable for enterprise-level solutions and therefore the actual operational VOPS would 
require a more robust database. Nevertheless, the concepts are the same. The data will be 
stored in a database and accessed by the design architecture’s middle tier on its way to 
the presentation layer. For this reason, we will not delve too deeply into the mechanics of 
Access or the notional database we created. 
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D, THE VOPS PROTOTYPE 

Our original intent for the VOPS prototype was to mimic the actual Naval 
Personal Command’s web site design and navigation to the point where we introduce our 
new procedures. To that end, we designed our pages as a scaled down version of the 
actual Personal Command’s web site. 

On the BUPERS homepage (Figure 11), a link takes a user to the Selection Board 
page where all active boards are listed and hyperlinks take the user to the particular board 
he or she is interested in. (Figure 12). 



Figure 11. Simulated homepage for BUPERS 
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Figure 12. Selection Board Display Page 

While all current boards are listed, only boards that have information will have 
links. The purple link in figure 12 indicates a followed link which takes the user to 
Figure 13. 
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BBS 


I '3 http://IocaIhost/Visual/fyboards.asp - Microsoft Internet Explorer 


RIe Edit View Favorites Tools Help ^ 

Q Back ^ ^ ^ ^ Google Capt G MSN NPS mail NPS Marines BB ^ EB USMC 


A^ess http://localhost/Visual/fyboards.asp 




N/^VY PEKSOPSNEL 
eOMM/^lND 


A^COM 

Naval 


Bureau of 
Naval Personnel 


Welcome to the new improved Visual Online Promotion System (\'OPS). 

ACirVT DUTY 0-5 LINE B0.4RD 
Board ID# is 230 

THE BOARD HAS ADJOLH^TD 
Click here to login as a Board Member 


FY2004 

FY2003 

EHsible LCDRs uodatfd pS 0? 

Eligible LCDRs 

Membefshio 

PrprM5t Siinn1«nMWa1 Gitidanrf 

Release Process 

Membershio 

Precept 

Release Process 

Select Messape 

Selection Board Statistics 

Promotion Pav Increment Plan 

DUE TO THE SEXER WEATHER CONDITIONS IN THE NORTHEAST AND SOME 




O Trusted sites 


Figure 13. Individual Board Display Page 


So far, we have not introdueed any ehanges to the current website. In our 
prototype, anyone can access the pages we have seen thus far; it is only at Figure 13 that 
we branch off to provide dual capability for the candidate and the board member. 

In Figure 13, a board member can access the promotion board pages by clicking 
on the “Click here to login as a board member” link which will take him or her to Figure 
14. Regardless of the security approach adopted, this log in page will provide a level of 
security to the process necessary to protect the promotion selection process. 
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Figure 14. VOPS Board Member Log In Screen 

Once the user has been identified and verified, the system displays only the 
records he or she is assigned (Figure 15). The list of candidates will differ from board 
member to board member depending on the individual records assigned to each. With the 
click of a link, the board member has access to each candidate’s Officer Summary 
Record (OSR, Figure 16) Cover Page and Performance Summary Record (PSR, Figure 
17) which are both dynamic, web-enabled versions of the documents currently used to 
display the candidate’s history. We believe that providing a layout similar to what has 
been traditionally provided was necessary for procedural continuity so that the board 
members can access detailed information for their pre-tank record review. 
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N/^VY PEKSOPSNEL 
COMM/^tND 


iLcOM 

Naval 


Bureau of 
Naval Personnel 





Visnal Online Promotion System (\'OPS) 



CNP 

Gni dance 
2003 

Last >'ame 

First Name 

\n 

SSN 



Cherry' 

Richard 

NMI 

345678901 

OSR Cover 

PSR 

Main Pase 

Countryman 

Autumn 

M 

789012345 

OSR Cover 

PSR 


Ferrando 

Mike 

P 

678901234 

OSR Cover 

PSR 


Ford 

William 

P 

456789012 

OSR Cover 

PSR 


Grose 

Jason 

D 

123456789 

OSR Cover 

PSR 


Jones 

Karen 

A 

567890123 

OSR Cover 

PSR 


Jones 

John 

P 

045678901 

OSR Cover 

PSR 


Leon 

Jesus 

W 

890123456 

OSR Cover 

PSR 


Rogers 

Glenn 

A 

234567890 

OSR Cover 

PSR 


Smith 

Sb- 

S 

000123456 

OSR Cover 

PSR 


Done 


O Trusted sites 


Figure 15. Candidate Record Display for Individual Board Member 
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Figure 16. VOPS OSR Cover Page 
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Figure 17. VOPSPSRPage 


After clicking on the “Fitreps” link in Figure 17, the user is taken to a dynamic 
page that displays summary performance marks for the candidate in the same manner the 
tank currently sees the information. Because the reviewer’s needs differ slightly from the 
tank needs, this traditional view of information is necessary for the reviewer. Because the 
tank members benefit from summary, graphical information rather than raw data, we 
redesigned the fitness report data to take advantage of the visual techniques discussed in 
chapter Ill (Figure 18). 
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Figure 18. Dynamic Fitness Report Overview Graph 

In our design, the data is normalized so that a reporting senior’s (RS) grading 
behavior is neutralized. For example, if a candidate performs extremely well at his first 
duty station, but because his reporting senior tends to grade everyone toward the bottom 
of the performance scale, the candidate might receive a grade of 3.0 which is above 
average for that reporting senior. At his next command, the candidate might do very 
poorly but the reporting senior grades high with a result of a 4.0 which translates to a 
below average mark from that particular reporting senior. Looking at these two grades 
without normalizing, a board member will interpret that the candidate performed poorly 
at his first command and better at his second command; exactly the opposite of the 
reality. 

For this reason, we normalized the data, taking into account the grading history of 

each reporting senior. In Figure 18, a grade of 1.0 is the exact average of the reporting 

senior so anything above that line indicates superior performance, and where anything 

below 1.0 indicates poor performance, relative to the reporting senior grading criteria. In 
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the design of these graphics, we tried to incorporate the important principles of graphical 
integrity as spelled out in chapter three. “Graphics must not quote data out of context ... 
the representation of numbers, as physically measured on the surface of the graphic itself, 
should be directly proportional to the numerical quantities represented” (Tufte, 2001). 

Similar to the fitness report display, we designed a page that dynamically pulls 
award information and displays it to the reviewer. Figure 19 is an example of the new 
awards page and shows the board members at a glance not only the awards the candidate 
received, but the relative importance of each award, indicated by the height of the bar; the 
higher the award precedence, the taller the bar. In this way, the board spends less time 
correlating the different awards and can see when each award was earned. 


'3 http://IocaIhost/Visual/dap_Awards4.asp 

- Microsoft Internet Ej^Iorer 

BB® 

RIe yit \rtew Favorites Tools Help 


It 

©Batk'^ 

Links Goo^e Capt G MSN NP5 mail IMPS Marines .(^ BB EB USMC 


A^ess http://localhost/Visual/dap Awards4,asp 


V 

1 1- 


0 

CNP 

Cin dance 
2003 


lain Page 


< 

Done 


SSN 

123456789 


Sum of Importance 


1/1/1985 1/12/1985 1/12/1986 1/12/1987 1/12/1988 1/12/1989 

Awards 


O Trusted sites 


Figure 19. Dynamic Awards Graph 


By providing the user with individual charts, the prototype enables both the 
reviewer and the board members with specific information concerning each candidate. 
But the vision of the VOPS was to aid the user in quickly and efficiently interpreting a 
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range of promotion factors. With this in mind, the system provides a combined chart 
(Figure 20) that offers a comparative correlation over a common timeline. With this 
disparate data collected into one chart, the board members can easily achieve 
comparative analysis at a glance. As Tufte writes. 

We envision information in order to reason about, communicate, 
document, and preserve that knowledge - activities nearly always carried 
out on two-dimensional paper and computer screen. Escaping this flatland 
and enriching the density of data displays are the essential tasks of 
information design (Tufte, 1990). 

Using this concept of information density, each layer extracts data and translates 
it into a graphical output on the common timeline. A briefer has the option to choose 
which graphs he or she wants to display and also has the capability to drill down for more 
detail by simply clicking on the graphs. 

The system also provides tools for the briefer to highlight important aspects of the 
candidate’s career by using the mouse to “draw” on the screen. 

The idea of the display capability is to provide the briefer with a set of tools he or 
she can choose from while preparing the presentation prior to the tank sessions. The 
precept could dictate common presentation rules but within those boundaries, the briefer 
has the capability to organize a specialized brief, complete with highlights and optional 
displays, for each record. 

In Figure 20, the top line is an overview of the candidate’s sea duty (blue bar) 
versus shore duty (brown bar). The next layer is the candidate’s promotion 
recommendations drawn from the PSR information which correlates to the performance 
recommendations, or the “fitrep” overview, displayed on the third tier. Considered 
individually, the layers convey trend lines over time. Taken together, the board members 
can easily correlate different aspects of the candidate’s record over a common timeline 
and can more easily make associations to certain data. 

The awards data, similar to the data displayed in Figure 19, appears on the fourth 
level where we’ve added a color coding system that aids in award type differentiation 
where the eye can easily follow a vertical line that associates the award with a specific 
time period. 
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The final layer shows the duty assignment overview as a baseline where the board 
can get a common understanding of the unit the candidate was working at in relation to a 
time dimension. Additionally, the system provides the candidate’s billet assignment 
above the command he or she was assigned to. This information is vital when the board 
sees the candidate’s display because they are able to match the type and duration of a 
specific billet with corresponding timeframes. 



Figure 20. Consolidated Promotion Data Page 
In our first iteration of the prototype, we did not provide the duty assignment 
information. After feedback from the Naval Personnel Command, we discovered a need 
to further specify not only the unit that the candidate belonged to, but what his or her job 
title was during the tour. We therefore designed an improved duty layer that showed both 
the command and duty assignment over the common timeline, as seen at the bottom of 
Figure 20. 
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E, SUMMARY 

Regardless of the approaeh eventually adopted, the VOPS prototype ean be 
implemented via an Internet or Intranet solution. By using three-tier arehiteeture, the 
system provides a maintainable, stable, and robust capability and enables the users to 
quickly and simply display a large amount of promotion data in a superior interface. The 
improved presentation takes advantage of proven techniques outlined in chapter III and 
provides both the briefers and board members with powerful cognitive tools to analyze 
and compare the different promotion criteria for each candidate. The end result is less 
mental computation effort required of the board members and a quicker decision cycle. 
Together, these factors increase the possibility of quality decisions, translating in a better 
promotion system. 

Our ROK analysis has proven that by using a Web-enabled system and 
graphically displaying the information it would make the process more efficient saving 
time and money in each board session. In order to answer the first question of whether or 
not the new process is more effective or more objective we would propose using a side by 
side analysis where the new system is used in conjunction with the old system in an 
actual board. We would recommend that this board be a smaller administrative board 
where the participant board members would have the time and leeway to respond with 
insights and suggestions. By surveying the board members immediately after the tank 
session we could gain immediate and helpful insights as to weather or not the graphical 
displays assist in perceiving and correlating the quantitative information. They could 
provide invaluable information as to how to improve the test system as well. By 
surveying multiple boards and making the suggested improvements along the way it 
would be possible to acquire empirical feedback necessary to determine whether or not 
the proposed system contributes or detracts from increasing the effectiveness and 
objectiveness of the overall process. We believe that by reducing the preparation, 
briefing, and voting times for the briefers’, we can ultimately reduce some of the 
emotional and physical strain internal to multiple decisions in a confined time period. 
This improvement may well translate to better, more objective, and ultimately more 
effective and objective decisions. 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. INTRODUCTION 

This thesis examines the Naval Promotion system and how eurrent board 
proeedures display information to the deeision-makers. By examining proven data 
visualization methods and integrating information teehnology into the system, we explore 
the benefits realized by re-designing the procedures that currently require the board 
members to process raw data into useful information. 

This chapter provides the conclusions drawn from the research . It also provides 
recommendations for further study in the areas of IT implementation and visualization 
techniques. 

B, CONCLUSIONS 

The Naval selection board process has embedded procedures that have sufficed 
for years. By observing their methods, we discovered the possibility to make the working 
system even better based on modem IT and visualization theories. We concluded that 
there is no need for a drastic overhaul of the entire system but rather an incremental 
upgrade incorporated within their current procedures. Specifically, our thesis concludes 
that a relatively small investment in IT will result in significant returns for many aspects 
of the existing framework. 

1, Effectiveness and Objectiveness of the Selection Board Process 

Our primary thesis question asked if a data-information visualization best 
practices can be embedded in information technology in order to improve the overall 
effectiveness and objectiveness of the selection board process. Using the Knowledge 
Value Added (KVA) methodology as a metric, we concluded that significant gains are 
possible using data-to-information visualization best practices embedded in information 
technology. Our results show a 126.52 percent increase in ROK. by introducing our 
recommendations. We can conclude from the increase in ROK that the efficiency of the 
“to-be” process would be significantly improved from the changes. By using a fully 
web-enabled system, it would reduce process instmctions involved in the sub - processes 
that take place later stages of the boards overall process which would result in an overall 
more efficient process. Further, we can surmise that by using our approach, the 
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execution time in hours would decrease by 120 hours for every 1000 records briefed in 
the Briefer Review/Annotation phase. Additionally, the “to-be” model showed a 
reduction of 60 hours per 1000 records in the Brief/PresentationA^ote phase. Further, we 
believe that by implementing the data visualization techniques introduced in this thesis it 
would shorten the process instruction involved in the “voting” process even more due to 
its powerful ability to convert data objects into information so quickly. 

Long term effects could include an increase in retention rates due to the 
enhancement of personnel satisfaction attributed to a more effective determination of the 
future promotion or assignment of Navy personnel. Unfortunately there is no way of 
knowing or accurately measuring the validity of this assumption. However, it is 
reasonable to conclude that by decreasing the strain on the board members and the time it 
takes to mentally assimilate the data required to make a timely decision that this would 
ultimately result in an overall increase in the boards effectiveness especially in the later 
stages of the board when board members are more susceptible to burn out or fatigue. 

The design of these displays should strive for uniformity in graphics and 
assurance that the representation accurately depicts the target numbers. We believe that 
the objectivity of the current system will be maintained or enhanced by designing the 
graphical display in such a manner that they accurately portray the information that is 
given. The ultimate design of the system should keep in mind Tufte’s guidance that the 
representation of numbers, as physically measured on the surface of the graphic itself, 
should be directly proportional to the numerical quantities represented. Additionally, to 
gain the most objective results a clear, detailed, and thorough labeling should be used to 
defeat graphical distortion and ambiguity. Write out explanations of the data on the 
graphic itself Label important events in the data. 

2, Time Reduction 

We concluded that integrating web-based technology with the visualization 
techniques will reduce the time required to prepare, brief, and make decisions in the 
promotion process. The current process uses a proprietary display system that simply 
projects pictures of paper-based records and a crude interface of summary data. By 
offering a web interface, we conclude that the same data drawn from a database and 

displayed in graphical forms or organized in a system of hyperlinks will reduce the time 
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required of all participants. The preparer spends less time correlating the data for the 
board members which shortens his or her brief. More importantly, the system enables the 
board members to make quicker decisions based on information more easily ingested. 

3. Appropriate Technologies 

Web-based data visualization technologies can serve as an enabler to this problem 
but the question remains of what are the relevant criteria for deciding which technologies 
are the most appropriate. 

Based on our research and experience with this problem, we conclude that a three- 
tier architecture is most appropriate. The robust nature and the current maturity of the 
architecture lends itself to storing, processing, and presenting information needed for the 
promotion boards. Using this layered approach, technology advances can be adopted in 
each layer thus providing expandability to the system without requiring complete 
dependency on current technology maturity in the entire system. 

4, User Interface Design 

What is an appropriate user interface design for this decision problem? We 
believe that a web-based interface provides a clear, concise, familiar interface for the 
modem board member. Using graphs generated by the data, the board members can 
quickly assimilate the data and make a decision. As the members become accustomed to 
the format, time required to locate pertinent data decreases. By using the interface most 
prevalent in personal and professional computer interaction, we take advantage of a built- 
in skill set to the computer-savvy members. Even for those less computer-savvy, the 
interfaces are intuitive and the graphs adhere to the proven theories set forth by Tufte, 
thus benefiting all the board members regardless of their technical background. 

C. RECOMMENDATIONS FOR FURTHER STUDY 

1. Implementation of the Theoretical Study 

Within the next year is an optimal timeframe to carry forward the research 
outlined in this thesis. The Military Personnel Command, in conjunction with the Space 
and Naval Warfare Systems Information Technology Center (SITC), developed the 
Single Integrated Human Resources Strategy (SIHRS). The goal of this strategy is to 
support the Defense Integrated Military Human Resources System (DIMHRS). DIMHRS 
is the Joint program, also initiated from the 1996 Defense Science Board, which will 
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unify human resource support for both active duty and reserve members of all four of the 
armed services (Feldmann, 2003). Additionally, at this time Naval Personnel Command 
is implementing a technical up-grade on the EMPRS system and the goal is that it will all 
be Web-based in the near future. We recommend collaborative involvement with NPC to 
incorporate our theoretical solution into the evolving upgrades. This thesis provides 
academic support based on proven visualization methods therefore the next logical step is 
to move forward with NPC for the implementation phase. Important factors to take into 
account are the following: 

• compatible with EMPRS 

• compliant within the NMCI network standards 

• compatible with PeopleSoft HRMS applications 

2. Five-Vector Model Incorporation 

Eventually a system such as this could be incorporated to display the Eive Vector 
Model where individuals would be evaluated for promotion and advancement throughout 
their career as outlined by the vector model. The Eive Vector Model aligns professional 
development requirements with certifications and qualifications both military and civilian 
that are recognized as the industry standard. Once the fleet has identified the 
requirements, subject matter experts in each field of expertise plot them on the vector 
model where an individual should ideally meet those milestones in their career. The 
model also considers personal development, leadership and management, and individual 
performance. By using the measures of performance in the Eive Vector Model, it may 
eventually be possible to eliminate the correlation of OSR/PSR data in the promotion 
process entirely. 

Implementing our design using the model as a baseline template is more of a 
radical redesign but something that PERS-8 has a definite interest in. We recommend 
having one vector on one screen (five vectors for the five screens). Using this 
methodology, the board would not even have to use the OSR/PSR data but rather gather 
the information they need based on the combination of visualization techniques outlined 
in this thesis and the Eive Vector Model. Eurthermore, by directly extracting the data for 
the Eive Vector Model from Naval Personnel Database or NSIPS (Navy Standard 
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Integrated Personnel System whieh will replace 4 legacy systems used in the Fleet), a 
single source solution will be possible. 

3, Other Services 

The ideas and prototype we developed for this thesis are portable to the other 
services. Minor personalization for each branch would be necessary but implementing the 
system could improve all the board operating procedures. We recommend research into 
the other services’ promotion processes and an evaluation of each to determine how this 
thesis could improve each process. We feel that such an evaluation would reveal 
tremendous possibilities to improve any and all promotion boards. 

4, Virtual Board 

The most radical recommendation we have is a study to determine if a virtual 
board could eventually occur. Due to the rapidly advancing technologies of collaborative 
tools and database utilities, combined with high bandwidth and reliable, secure 
connections, we feel that the Navy could save a large amount of funds by conducting 
boards in a disbursed, collaborative manner. 

Our vision is to have board members log on via secure accounts and perform their 
record evaluations online. The centralized hub could take care of the administration and 
logistics but massive savings could be realized by not requiring board members to travel 
to a common site for the proceedings. Whether the board would resemble a Video 
Teleconference (VTC) or a prerecorded presentation of each record brief are some of the 
issues we recommend for further study. 
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